wiki:ComponentRegistryRest

Component registry REST interface

In an effort to disclose the Component Registry by means of web services an REST interface has been developed.

The REST interface is online. The Rest API which is on the server allows a client (e.g the React front end) to browse and retrieve all existing components and profiles. Example:

If properly authenticated, users can register new ones to their workspace and eventually publish them to the ​public registry.

Interactive documentation of the REST service can be accessed at ​https://catalog.clarin.eu/ds/ComponentRegistry/api-docs.

Contents

  1. Component registry REST interface
    1. Contents
    2. Getting code
    3. Building and Deploying
      1. Building
      2. Deployment
      3. Dependencies
    4. Design
      1. Class diagrams
      2. Data and service layer coupling
      3. Authentication
    5. Usage
      1. Testing environment
    6. Docker

Getting code

  • You can browse/get/fork the code ​here.

Building and Deploying

Building

The Component Registry REST service/back end is written in Java and can be built through its Maven project ​component-registry-rest. The source targets Java 11, so this is the minimal version of Java required to build and a compatible version of Maven is required as well. Build output includes a WAR file that can be deployed in Tomcat, but also a deployment package that includes documentation and database scripts. Note that there are several build profiles; it is important to select the right one for your deployment use case.

The project can be built and installed in the local repository like any Maven project by running mvn clean install. It also comes with a build script (build.sh) that carries out this build in a Docker container so that the required versions of Java and Maven do not have to be installed on the host.

An automated build is also configured on Travis (​travis-ci.org/clarin-eric/component-registry-rest). Pushing changes to GitHub will trigger a build; pushing a tag will trigger a deployment of the build output (deployment package) to GitHub. This deployment package can be used to build a Docker image for the back end (see below).

See the ​component-registry-rest repository for additional information.

Maven build

Executing the package phase of the project will generate a WAR file that can be deployed on Tomcat (or potentially another servlet container (UNTESTED)). The build process includes a call to the ​maven-jaxb2 plugin, which generates model classes from the component schema from the ​cmdi toolkit. The entire code base depends on these classes, so any code checker will detect errors until at least one build has been executed.

Build profiles

The ​pom defines a number of profiles:

  • default: usable default settings, suitable for local deployment; basic authentication, not for production deployment
  • development configures logging for development: debug level output to console and localhost access; basic authentication
  • docker builds prepared for Docker image build (​docker-component-registry-rest), puts placeholders in place that are filtered build time or run time on through environment variables and/or arguments; configurable authentication
  • jar builds a jar so that the Component Registry can be used as a dependency in other Java projects
  • readonly builds a version that has no options for authentication so that no changes to the data can be made
  • DEPRECATED clarin build for native production deployment in the CLARIN infrastructure

Deployment

First, build the application with the profile according to the targeted environment (see above). For most deployment cases within CLARIN, this will be the docker profile. See Docker section below.

Tomcat deployment

For manual deplyoment into a Tomcat servlet container:

  • To deploy the REST service including the front end, deploy the build output (WAR file) of the REST service to a Tomcat instance.
  • The application needs a Postgres database connection configured for it to actually work. If a database does not exist yet, it can be created using the ​create.sql script. By default, the application assumes a named (JNDI) datasource jdbc/ComponentRegistry that provides access to the Component Registry database. An example is provided in ​spring-config/datasource-jdbc.xml. This should ideally be configured in the server's global context.xml but can also be done in the servlet context.
  • More configuration options are available in ​spring-config/component-registry.xml; generally these will not need much adaptation; you'll have to restart the application after changing any of these.

Dependencies

Some major external dependencies:

  • Spring framework (version 5.1 is currently used)
  • Jersey
  • PostgreSQL
  • HSQLDB (for testing)
  • Wicket (for the admin interface)
  • ...

The REST service also depends on

  • the ​CMDI toolkit, specifically the component schema, which is used for validating component and profile specifications, and the comp2schema XSLT, which is used to generate XSD's from profile specifications
    • Locations of current toolkit schemas and stylesheets
  • ​CMDValidate for validation (XSD and Schematron) of component specifications
  • External services:
    • Concept registry (CCR)
    • Vocabulary service (CLAVAS)

Design

This project is mainly written in Java with Jersey (JAX-RS implementation). However, it also contains some other code that implements a simple administration interface. The front end is defined in a separate project (see ComponentRegistryAndEditor). The below diagrams show the hierarchical structure of the Component Registry Rest service and provide a better overview of the project.

Class diagrams

Note: these diagrams are outdated and do not strictly represent the current application design accurately

Rest service Diagram

Figure 1. rest package diagram

Registry diagram

Figure 2. registry package diagram

The rest interface is implemented using Jersey on top of the server, most importantly an implementation of the ComponentRegistry interface.

Front End diagram

Figure 3. front end package diagram

Implementation Database Diagram

Figure 4. implementation database package diagram

Data and service layer coupling

The implementation of this service layer is using a PostgreSQL database to store the components and profiles. The project uses the Spring framework. First of all, it uses its DAO support to interact with the database (*Dao classes in Figure 4)

Spring provides Dependency Injections methods used in the "wiring" of the application. This can be found in the ​configuration classes.

Authentication

The REST service is designed to use Shibboleth for authentication. Although other kinds of authentication should also work. For testing, simple Tomcat authentication can be used, see ​web.xml for a configuration without shibboleth (cf. ​web-shib.xml).

Since a Shibboleth session can only be initiated through user interaction within the browser, all clients will need to share this browser session. For the React front end this is trivial because it lives inside the browser and all requests go through the browser.

There is also a configuration that disables all authentication and thus effectively puts the Component Registry in read-only mode.

Model Diagram

Figure 5. model package diagram

Usage

For an overview of all calls, see the ​WADL or the ​interactive REST documentation.

Testing environment

An environment for testing purposes has been deployed (currently at a HostEurope? root server), accessible via ​https://catalog.clarin-dev.eu/ds/ComponentRegistry

Obviously, one should not rely on any data in this testing environment

Docker

A docker image can be built that takes the compiled REST project and bundles it in an Tomcat servlet based image ready for deployment. It requires some configuration, mainly:

  • Access context (public URL)
  • Database URL (needs to be deployed separately)
  • External service URLs
  • Authentication mode
    • Shibboleth settings if applicable
  • Administration users

For details, see ​docker-component-registry-rest.

Note: the REST service nor its image provide a front end, nor do they depend on it. The front end has a ​separate image which can be managed together with the back end and a databae instance to provide a complete solution using a dedicated ​Compose project.

Last modified 4 years ago Last modified on 11/15/19 09:31:16

Attachments (7)

Download all attachments as: .zip