= Component registry REST interface = In an effort to disclose the [[ComponentRegistryAndEditor|Component Registry]] by means of web services an REST interface has been developed. The REST interface is online. The Rest API which is on the server allows a client (e.g the [[ComponentRegistryAndEditor/ReactFrontEnd|React front end]]) to browse and retrieve all existing components and profiles. Example: * get all components: [http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/components] * get all profiles: [http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/profiles] If properly authenticated, users can register new ones to their workspace and eventually publish them to the [http://catalog.clarin.eu/ds/ComponentRegistry public registry]. '''Interactive documentation''' of the REST service can be accessed at [https://catalog.clarin.eu/ds/ComponentRegistry/api-docs]. ---- == Contents == [[PageOutline(1-3, , inline)]] ---- == Getting code == * You can browse/get/fork the code [https://github.com/clarin-eric/component-registry-rest here]. ---- == Building and Deploying == === Building === The Component Registry REST service/back end is written in Java and can be built through its Maven project [https://github.com/clarin-eric/component-registry-rest component-registry-rest]. The source targets **Java 11**, so this is the minimal version of Java required to build and a compatible version of Maven is required as well. Build output includes a WAR file that can be deployed in Tomcat, but also a deployment package that includes documentation and database scripts. Note that there are several build profiles; it is important to select the right one for your deployment use case. The project can be built and installed in the local repository like any Maven project by running `mvn clean install`. It also comes with a **build script** (`build.sh`) that carries out this build in a Docker container so that the required versions of Java and Maven do not have to be installed on the host. An **automated build** is also configured on [[TravisCI|Travis]] ([https://travis-ci.org/clarin-eric/component-registry-rest travis-ci.org/clarin-eric/component-registry-rest]). Pushing changes to GitHub will trigger a build; pushing a tag will trigger a deployment of the build output (deployment package) to GitHub. This deployment package can be used to build a Docker image for the back end (see below). See the [https://github.com/clarin-eric/component-registry-rest component-registry-rest repository] for additional information. ==== Maven build ==== Executing the ''package'' phase of the project will generate a WAR file that can be deployed on Tomcat (or potentially another servlet container (UNTESTED)). The build process includes a call to the [http://java.net/projects/maven-jaxb2-plugin/pages/Home maven-jaxb2 plugin], which generates model classes from the component schema from the [https://github.com/clarin-eric/cmdi-toolkit cmdi toolkit]. The entire code base depends on these classes, so any code checker will detect errors until at least one build has been executed. ===== Build profiles ===== The [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/pom.xml pom] defines a number of profiles: - default: usable default settings, suitable for local deployment; basic authentication, not for production deployment - **development** configures logging for development: debug level output to console and localhost access; basic authentication - **docker** builds prepared for Docker image build ([https://gitlab.com/CLARIN-ERIC/docker-component-registry-rest docker-component-registry-rest]), puts placeholders in place that are filtered build time or run time on through environment variables and/or arguments; configurable authentication - **jar** builds a jar so that the Component Registry can be used as a dependency in other Java projects - **readonly** builds a version that has no options for authentication so that no changes to the data can be made - `DEPRECATED` **clarin** build for native production deployment in the CLARIN infrastructure === Deployment === First, build the application with the '''profile''' according to the targeted environment (see above). For most deployment cases within CLARIN, this will be the `docker` profile. See Docker section below. ==== Tomcat deployment ==== For manual deplyoment into a Tomcat servlet container: * To deploy the REST service including the front end, deploy the build output (WAR file) of the REST service to a Tomcat instance. * The application needs a Postgres database connection configured for it to actually work. If a database does not exist yet, it can be created using the [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/sql/create.sql create.sql] script. By default, the application assumes a named (JNDI) datasource ''jdbc/ComponentRegistry'' that provides access to the Component Registry database. An example is provided in [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webresources/WEB-INF/spring-config/datasource-jdbc.xml spring-config/datasource-jdbc.xml]. This should ideally be configured in the server's global context.xml but can also be done in the servlet context. * More configuration options are available in [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webresources/WEB-INF/spring-config/component-registry.xml spring-config/component-registry.xml]; generally these will not need much adaptation; you'll have to restart the application after changing any of these. === Dependencies === Some major external dependencies: * Spring framework (version 5.1 is currently used) * Jersey * PostgreSQL * HSQLDB (for testing) * Wicket (for the admin interface) * ... The REST service also depends on * the [https://github.com/clarin-eric/cmdi-toolkit CMDI toolkit], specifically the ''component schema'', which is used for validating component and profile specifications, and the ''comp2schema'' XSLT, which is used to generate XSD's from profile specifications * Locations of current toolkit schemas and stylesheets * [https://github.com/clarin-eric/cmd-validate CMDValidate] for validation (XSD and Schematron) of component specifications * External services: * Concept registry (CCR) * Vocabulary service (CLAVAS) ---- == Design == This project is mainly written in Java with Jersey (JAX-RS implementation). However, it also contains some other code that implements a simple administration interface. The front end is defined in a separate project (see [[ComponentRegistryAndEditor]]). The below diagrams show the hierarchical structure of the Component Registry Rest service and provide a better overview of the project. === Class diagrams === '''Note:''' these diagrams are ''outdated'' and do not strictly represent the current application design accurately [[Image(clarin.cmdi.componentregistry.rest.png)]] Figure 1. rest package diagram [[Image(clarin.cmdi.componentRegistry.png)]] Figure 2. registry package diagram The rest interface is implemented using Jersey on top of the server, most importantly an implementation of the ComponentRegistry interface. [[Image(clarin.cmdi.componentregistry.frontend.png)]] Figure 3. front end package diagram [[Image(clarin.cmdi.componentregistry.impl.database.png)]] Figure 4. implementation database package diagram === Data and service layer coupling === The implementation of this service layer is using a PostgreSQL database to store the components and profiles. The project uses the Spring framework. First of all, it uses its DAO support to interact with the database (*Dao classes in Figure 4) Spring provides Dependency Injections methods used in the "wiring" of the application. This can be found in the [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webresources/WEB-INF/ configuration classes]. === Authentication === The REST service is designed to use Shibboleth for authentication. Although other kinds of authentication should also work. For testing, simple Tomcat authentication can be used, see [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webapp/WEB-INF/web.xml web.xml] for a configuration without shibboleth (cf. [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webapp/WEB-INF/web-shib.xml web-shib.xml]). Since a Shibboleth session can only be initiated through user interaction within the browser, all clients will need to share this browser session. For the [[ComponentRegistryAndEditor/ReactFrontEnd|React front end]] this is trivial because it lives inside the browser and all requests go through the browser. There is also a configuration that disables all authentication and thus effectively puts the Component Registry in read-only mode. ---- [[Image(clarin.cmdi.componentregistry.model.png)]] Figure 5. model package diagram ---- == Usage == For an overview of all calls, see the [http://catalog.clarin.eu/ds/ComponentRegistry/rest/application.wadl WADL] or the [https://catalog.clarin.eu/ds/ComponentRegistry/api-docs interactive REST documentation]. === Testing environment === An environment for **testing purposes** has been deployed (currently at a HostEurope root server), accessible via https://catalog.clarin-dev.eu/ds/ComponentRegistry * get all components: [https://catalog.clarin-dev.eu/ds/ComponentRegistry/rest/registry/components] * get all profiles: [https://catalog.clarin-dev.eu/ds/ComponentRegistry/rest/registry/profiles] Obviously, one should not rely on any data in this testing environment == Docker == A docker image can be built that takes the compiled REST project and bundles it in an Tomcat servlet based image ready for deployment. It requires some configuration, mainly: * Access context (public URL) * Database URL (needs to be deployed separately) * External service URLs * Authentication mode * Shibboleth settings if applicable * Administration users For details, see [https://gitlab.com/CLARIN-ERIC/docker-component-registry-rest docker-component-registry-rest]. Note: the REST service nor its image provide a front end, nor do they depend on it. The front end has a [https://gitlab.com/CLARIN-ERIC/docker-component-registry-front-end separate image] which can be managed together with the back end and a databae instance to provide a complete solution using a dedicated [https://gitlab.com/CLARIN-ERIC/compose_compreg Compose project].