Author/responsible: Twan
Guidelines for software development in projects
This documents contains guidelines for software development within CLARIN projects. Since such projects generally have a limited temporal scope, and the original developer(s) will often not be available for support outside the context of the project, additional care has to be taken with respect to a number of aspects that factor into the 'sustainability' of a software project. Providing good documentation, choosing commonly understood and well supported technologies, guarding quality throughout the development process, and ensuring the portability of the resulting code base strongly increase the chances of a piece of software surviving the end of its "parent project".
We (CLARIN) suggest that you commit to these guidelines if you are working on a software project in the context of a project. You can do this explicitly by having this document referenced in the relevant task description(s).
Please do not hesitate to contact the maintainer of this page if you have feedback or questions. Any remarks, suggestions or other feedback are more than welcome!
Documentation
Arguably the most important aspect of making software survive is providing good documentation. Different types of documentation should be produced and maintained throughout the various stages of software development. For these guidelines, we are mainly concerned with documentation aimed at developers and system administrators, although many applications also require a user manual.
- Make sure that a description of the requirements of the developed software is available. This will often be part of the project description, but this description may not always be sufficiently descriptive. Ideally, functional and technical design documents are prepared and made available.
- Provide a description of the architecture of the software, either in prose or by means of (a) UML diagram(s)
- Provide good quality in-code documentation. Follow the conventions associated with the language, framework or platform that you are using (e.g. use Javadoc for Java rather than just inline comments).
- If applicable, document your API, especially if its public facing. There are tools that make this easy, such as Swagger (for Java)
- Bundle some basic documentation with your sources 'and' distribution packages. Include at least the following:
README(brief project description, preferably with pointers to more detailed information)- If required, include build instructions (ideally these should be very short or even redundant)
INSTALL(setup and configuration instructions)LICENSE(source code licencing information)
- Make a wiki page in the CLARIN Trac for the software project or make a section for it in another page if that is more appropriate. This page can serve as a hub to find sources and technical, administrative and other information. Add a link to the page to the software development page.
- Make use of an issue tracking system, such as the CLARIN Trac or the 'Issues' section of a GitHub repository. Make separate tickets for all known issues as well as potential enhancements. Use milestones or priorities to indicate which issues should be addressed first or which features should be included in a single release.
Implementation decisions
Or how to choose a programming language, development frameworks and tools.
- If possible, select a programming language that is widely known and already used within the CLARIN infrastructure, an in particular CLARIN's core development team. For applications that run on a server, Java is the preferred language of choice. For web applications front-ends that run client-side, Javascript is used preferably combined with one of the frameworks mentioned below. Scripts should preferably be written in Python, or alternatively in Bash.
- The same applies to any frameworks that you might use to develop your application. The following frameworks are preferred for various programming languages/environments:
- Apache Wicket for Java-based web-applications
- JAX-RS/Jersey for REST services
- React for Javascript based front-ends
- Bootstrap for web page/front-end styling (HTML & CSS)
- Use the Bootstrap based CLARIN base style to match the common CLARIN styling
Quality and integrity
Software quality is an extremely broad and multi-faceted topic. Quality is foremost determined by the degree to which a software meets its functional requirements, which will naturally differ per project. This section of the guidelines is concerned with general measures aimed at easing the potential transfer of (the maintenance of) a code base from one developer to another. See Software Quality on Wikipedia.
- Follow the conventions with respect to code layout, naming and project structure/organisation associated with the language and/or framework that you are using. There may not always be 'official' or uniform conventions, but Google has style guides for various languages (including Java, Javascript, Python and Shell) that generally provide good guidelines. Obviously, the most important thing is to use consistent style.
- Provide automated tests (unit tests) for your source code. Various tools are available to measure the test coverage of your project. Cobertura is an open-source solution for Java projects.
- Run your tests every time you publish code or binaries. You can test locally on every build, or even better set up remote execution of your tests on a regular basis or triggered by commits/pushes using a continuous integration solution that hooks into your versioning control system (such as Travis).
- Try to get your code reviewed, either by a colleague or someone else within the CLARIN community. Start the reviewing process early on, as it usually gets exponentially harder to make structural changes to your code base while it grows.
- Make sure that there are no secrets such as passwords, or other environment specific values in your sources. Use external files (e.g. property files or XML documents) to provide these to your software and provide an example of such a file with your code.
Portability
See Software portability on Wikipedia.
- Use a 'modern' build automation tool and provide a build descriptor that does not rely on a highly specific context, i.e. make sure that your software can be built 'out of the box' if feasible, or provide instructions to prepare the environment for building the tool otherwise. Apache Maven is used most commonly (for Java) within CLARIN. Other options are Gradle (for Java and several other languages) and Grunt (for Javascript).
- Make sure that all dependencies are available, either via public repositories or via additional repositories defined in your build descriptor.
- Test your build regularly, ideally on a daily basis or with every commit/push, i.e. attempt to implement continuous integration. Generally applicable tools that can be used for this are, among others, Travis and Jenkins.
- Consider distributing your application as a Docker image. A good place to get started, is the CLARIN ERIC docker build workflow documentation. The Docker website also has instructions and more information.
- Make all sources available that are needed to build your software. Use GitHub for this purpose (you can join CLARIN's organisation).
Other
- Make yourself (and your team members) known and traceable. Use a recognisable handle on the various platforms (such as GitHub, Slack) and/or provide your real name in your profile. Make a personal page on the Wiki and add a link to it from the People page and add basic contact information.
