[[PageOutline]] Author/responsible: [[People/twagoo|Twan]] = Guidelines for software development in projects = This documents contains guidelines for software development within CLARIN projects. Since such projects generally have a limited temporal scope, and the original developer(s) will often not be available for support outside the context of the project, additional care has to be taken with respect to a number of aspects that factor into the 'sustainability' of a software project. Providing good documentation, choosing commonly understood and well supported technologies, guarding quality throughout the development process, and ensuring the portability of the resulting code base strongly increase the chances of a piece of software surviving the end of its "parent project". We (CLARIN) suggest that you commit to these guidelines if you are working on a software project in the context of a project. You can do this explicitly by having this document referenced in the relevant task description(s). Please do not hesitate to contact the maintainer of this page if you have feedback or questions. Any remarks, suggestions or other feedback are more than welcome! == Documentation == Arguably the most important aspect of making software survive is providing good documentation. Different types of documentation should be produced and maintained throughout the various stages of software development. For these guidelines, we are mainly concerned with documentation aimed at developers and system administrators, although many applications also require a user manual. * Make sure that a '''description of the requirements''' of the developed software is available. This will often be part of the project description, but this description may not always be sufficiently descriptive. Ideally, '''functional and technical design documents''' are prepared and made available. * Provide a '''description of the architecture''' of the software, either in prose or by means of (a) UML diagram(s) * Provide good quality '''in-code documentation'''. Follow the conventions associated with the language, framework or platform that you are using (e.g. use [http://www.oracle.com/technetwork/articles/java/index-137868.html Javadoc] for Java rather than just inline comments). * If applicable, '''document your API''', especially if its public facing. There are tools that make this easy, such as [http://swagger.io/ Swagger] (for Java) * '''Bundle''' some basic documentation with your sources 'and' distribution packages. Include at least the following: * `README` (brief project description, preferably with pointers to more detailed information) * If required, include build instructions (ideally these should be very short or even redundant) * `INSTALL` (setup and configuration instructions) * `LICENSE` (source code licencing information) * Make a '''wiki page''' in the [https://trac.clarin.eu CLARIN Trac] for the software project or make a section for it in another page if that is more appropriate. This page can serve as a hub to find sources and technical, administrative and other information. Add a link to the page to the [[SoftwareDevelopment|software development page]]. * Make use of an '''issue tracking system''', such as the [https://trac.clarin.eu CLARIN Trac] or the 'Issues' section of a GitHub repository. Make separate tickets for all known issues as well as potential enhancements. Use milestones or priorities to indicate which issues should be addressed first or which features should be included in a single release. == Implementation decisions == Or how to choose a programming language, development frameworks and tools. * If possible, select a '''programming language''' that is widely known and already used within the CLARIN infrastructure, an in particular CLARIN's core development team. For applications that run on a server, '''Java''' is the preferred language of choice. For web applications front-ends that run client-side, '''Javascript''' is used preferably combined with one of the frameworks mentioned below. Scripts should preferably be written in '''Python''', or alternatively in '''Bash'''. * The same applies to any '''frameworks''' that you might use to develop your application. The following frameworks are preferred for various programming languages/environments: * [http://wicket.apache.org/ Apache Wicket] for Java-based web-applications * [https://jax-rs-spec.java.net/ JAX-RS]/[https://jersey.java.net/ Jersey] for REST services * [https://facebook.github.io/react React] for Javascript based front-ends * [http://getbootstrap.com/ Bootstrap] for web page/front-end styling (HTML & CSS) * Use the Bootstrap based [https://github.com/clarin-eric/base_style CLARIN base style] to match the common CLARIN styling == Quality and integrity == Software quality is an extremely broad and multi-faceted topic. Quality is foremost determined by the degree to which a software meets its functional requirements, which will naturally differ per project. This section of the guidelines is concerned with general measures aimed at easing the potential transfer of (the maintenance of) a code base from one developer to another. See [https://en.wikipedia.org/wiki/Software_quality Software Quality on Wikipedia]. * Follow the '''conventions''' with respect to code layout, naming and project structure/organisation associated with the language and/or framework that you are using. There may not always be 'official' or uniform conventions, but Google has [https://github.com/google/styleguide style guides] for various languages (including [https://google.github.io/styleguide/javaguide.html Java], [https://google.github.io/styleguide/javascriptguide.xml Javascript], [https://google.github.io/styleguide/pyguide.html Python] and [https://google.github.io/styleguide/shell.xml Shell]) that generally provide good guidelines. Obviously, the most important thing is to use consistent style. * Provide '''automated tests''' (unit tests) for your source code. Various tools are available to measure the test coverage of your project. [http://cobertura.github.io/cobertura/ Cobertura] is an open-source solution for Java projects. * '''Run your tests''' every time you publish code or binaries. You can test locally on every build, or even better set up remote execution of your tests on a regular basis or triggered by commits/pushes using a continuous integration solution that hooks into your versioning control system (such as [[TravisCI|Travis]]). * Try to '''get your code reviewed''', either by a colleague or someone else within the CLARIN community. Start the reviewing process early on, as it usually gets exponentially harder to make structural changes to your code base while it grows. * Make sure that there are no '''secrets''' such as passwords, or other '''environment specific values''' in your sources. Use external files (e.g. property files or XML documents) to provide these to your software and provide an example of such a file with your code. == Portability == See [https://en.wikipedia.org/wiki/Software_portability Software portability on Wikipedia]. * Use a 'modern' '''build automation tool''' and provide a build descriptor that does not rely on a highly specific context, i.e. make sure that your software can be built 'out of the box' if feasible, or provide instructions to prepare the environment for building the tool otherwise. [http://maven.apache.org/ Apache Maven] is used most commonly (for Java) within CLARIN. Other options are [https://en.wikipedia.org/wiki/Gradle Gradle] (for Java and several other languages) and [http://gruntjs.com/ Grunt] (for Javascript). * Make sure that all '''dependencies''' are available, either via public repositories or via additional repositories defined in your build descriptor. * Test your build regularly, ideally on a daily basis or with every commit/push, i.e. attempt to implement '''continuous integration'''. Generally applicable tools that can be used for this are, among others, [[TravisCI|Travis]] and [https://jenkins.io/ Jenkins]. * Consider distributing your application as a '''[https://docker.com/ Docker] image'''. A good place to get started, is the [https://gitlab.com/CLARIN-ERIC/build-workflow-documentation CLARIN ERIC docker build workflow documentation]. The Docker website also has [https://docs.docker.com/get-started/ instructions and more information]. * Make '''all sources available''' that are needed to build your software. Use GitHub for this purpose (you can join [https://github.com/clarin-eric CLARIN's organisation]). == Other == * Make yourself (and your team members) known and traceable. Use a recognisable handle on the various platforms (such as !GitHub, Slack) and/or provide your real name in your profile. Make a personal page on the Wiki and add a link to it from the [[People|People page]] and add '''basic contact information'''. = Resources = * [https://technical-reference.readthedocs.io/en/latest EURISE technical reference] * [https://github.com/CLARIAH/software-quality-guidelines CLARIAH Software Quality Guidelines] * [https://github.com/google/styleguide Google styleguides]