Changes between Version 29 and Version 30 of ComponentRegistryRest


Ignore:
Timestamp:
11/15/19 09:31:16 (5 years ago)
Author:
Twan Goosen
Comment:

updated for 2.3

Legend:

Unmodified
Added
Removed
Modified

  • ComponentRegistryRest

    v29 v30  
    2525
    2626=== Building ===
    27 The REST is part of the modular Maven project [[ComponentRegistryAndEditor|ComponentRegistry]]. This module can be built separately ([https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/pom.xml pom]) but depends on the [[ComponentRegistryAndEditor/ReactFrontEnd|React front end]] so it needs to be installed in the local repository.
     27The Component Registry REST service/back end is written in Java and can be built through its Maven project [https://github.com/clarin-eric/component-registry-rest component-registry-rest]. The source targets **Java 11**, so this is the minimal version of Java required to build and a compatible version of Maven is required as well. Build output includes a WAR file that can be deployed in Tomcat, but also a deployment package that includes documentation and database scripts. Note that there are several build profiles; it is important to select the right one for your deployment use case.
    2828
    29 Executing the ''package'' phase of the project will generate a WAR file that can be deployed on Tomcat (or potentially another servlet container).
     29The project can be built and installed in the local repository like any Maven project by running `mvn clean install`. It also comes with a **build script** (`build.sh`) that carries out this build in a Docker container so that the required versions of Java and Maven do not have to be installed on the host.
    3030
    31 The build process includes a call to the [http://java.net/projects/maven-jaxb2-plugin/pages/Home maven-jaxb2 plugin], which generates model classes from the component schema from the [https://github.com/clarin-eric/cmdi-toolkit cmdi toolkit]. The entire code base depends on these classes, so any code checker will detect errors until at least one build has been executed.
     31An **automated build** is also configured on [[TravisCI|Travis]] ([https://travis-ci.org/clarin-eric/component-registry-rest travis-ci.org/clarin-eric/component-registry-rest]). Pushing changes to GitHub will trigger a build; pushing a tag will trigger a deployment of the build output (deployment package) to GitHub. This deployment package can be used to build a Docker image for the back end (see below).
    3232
    33 ==== Build profiles ====
     33See the [https://github.com/clarin-eric/component-registry-rest component-registry-rest repository] for additional information.
    3434
    35 The [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/pom.xml pom] has profiles for three deployment scenarios: default (local), lux16 (testing) and clarin (production). They only differ with respect to the authentication pre-configuration (non-!Shibboleth/Shibboleth with specific settings).
     35==== Maven build ====
     36
     37Executing the ''package'' phase of the project will generate a WAR file that can be deployed on Tomcat (or potentially another servlet container (UNTESTED)). The build process includes a call to the [http://java.net/projects/maven-jaxb2-plugin/pages/Home maven-jaxb2 plugin], which generates model classes from the component schema from the [https://github.com/clarin-eric/cmdi-toolkit cmdi toolkit]. The entire code base depends on these classes, so any code checker will detect errors until at least one build has been executed.
     38
     39===== Build profiles =====
     40
     41The [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/pom.xml pom] defines a number of profiles:
     42- default: usable default settings, suitable for local deployment; basic authentication, not for production deployment
     43- **development** configures logging for development: debug level output to console and localhost access; basic authentication
     44- **docker** builds prepared for Docker image build ([https://gitlab.com/CLARIN-ERIC/docker-component-registry-rest docker-component-registry-rest]), puts placeholders in place that are filtered build time or run time on through environment variables and/or arguments; configurable authentication
     45- **jar** builds a jar so that the Component Registry can be used as a dependency in other Java projects
     46- **readonly** builds a version that has no options for authentication so that no changes to the data can be made
     47- `DEPRECATED` **clarin** build for native production deployment in the CLARIN infrastructure
     48
    3649
    3750=== Deployment ===
    3851
    39 First, build the application with the '''profile''' according to the targeted environment (see above).
     52First, build the application with the '''profile''' according to the targeted environment (see above). For most deployment cases within CLARIN, this will be the `docker` profile. See Docker section below.
    4053
    41 To deploy the REST service including the front end, deploy the build output (WAR file) of the REST service to a Tomcat instance.
     54==== Tomcat deployment ====
    4255
    43 The application needs a Postgres database connection configured for it to actually work. If a database does not exist yet, it can be created using the [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/sql/create.sql create.sql] script. By default, the application assumes a named (JNDI) datasource ''jdbc/ComponentRegistry'' that provides access to the Component Registry database. An example is provided in [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webresources/WEB-INF/spring-config/datasource-jdbc.xml spring-config/datasource-jdbc.xml]. This should ideally be configured in the server's global context.xml but can also be done in the servlet context.
     56For manual deplyoment into a Tomcat servlet container:
    4457
    45 More configuration options are available in [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webresources/WEB-INF/spring-config/component-registry.xml spring-config/component-registry.xml]; generally these will not need much adaptation; you'll have to restart the application after changing any of these.
     58* To deploy the REST service including the front end, deploy the build output (WAR file) of the REST service to a Tomcat instance.
    4659
    47 For deployment for testing on lux16 see Usage -> Testing Environment in this document, i.e. [https://trac.clarin.eu/wiki/ComponentRegistryRest#Usage]
     60* The application needs a Postgres database connection configured for it to actually work. If a database does not exist yet, it can be created using the [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/sql/create.sql create.sql] script. By default, the application assumes a named (JNDI) datasource ''jdbc/ComponentRegistry'' that provides access to the Component Registry database. An example is provided in [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webresources/WEB-INF/spring-config/datasource-jdbc.xml spring-config/datasource-jdbc.xml]. This should ideally be configured in the server's global context.xml but can also be done in the servlet context.
     61
     62* More configuration options are available in [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webresources/WEB-INF/spring-config/component-registry.xml spring-config/component-registry.xml]; generally these will not need much adaptation; you'll have to restart the application after changing any of these.
    4863
    4964=== Dependencies ===
    5065
    51 Internal dependency (included build time): [[ComponentRegistryAndEditor/ReactFrontEnd|React front end]]
    52 
    53 External dependencies:
    54 * Spring framework (version 3.2 is currently used)
     66Some major external dependencies:
     67* Spring framework (version 5.1 is currently used)
    5568* Jersey
    5669* PostgreSQL
    5770* HSQLDB (for testing)
     71* Wicket (for the admin interface)
     72* ...
    5873
    5974The REST service also depends on
    6075* the [https://github.com/clarin-eric/cmdi-toolkit CMDI toolkit], specifically the ''component schema'', which is used for validating component and profile specifications, and the ''comp2schema'' XSLT, which is used to generate XSD's from profile specifications
     76  * Locations of current toolkit schemas and stylesheets
    6177* [https://github.com/clarin-eric/cmd-validate CMDValidate] for validation (XSD and Schematron) of component specifications
     78* External services:
     79  * Concept registry (CCR)
     80  * Vocabulary service (CLAVAS)
    6281----
    6382== Design ==
    6483
    65 This project is mainly build in Java. However, it also contains some other code that implements a simple administration interface and a page embedding the graphical client application. The below diagrams show the hierarchical structure of the Component Registry Rest service and provide a better overview of the project.
     84This project is mainly written in Java with Jersey (JAX-RS implementation). However, it also contains some other code that implements a simple administration interface. The front end is defined in a separate project (see [[ComponentRegistryAndEditor]]). The below diagrams show the hierarchical structure of the Component Registry Rest service and provide a better overview of the project.
    6685
    6786=== Class diagrams ===
    … …  
    97116=== Authentication ===
    98117
    99 The REST service is designed to use Shibboleth for authentication. although other kinds of authentication should also work. For testing, simple Tomcat authentication can be used, see [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webapp/WEB-INF/web.xml web.xml] for a configuration without shibboleth (cf. [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webapp/WEB-INF/web-shib.xml web-shib.xml]).
     118The REST service is designed to use Shibboleth for authentication. Although other kinds of authentication should also work. For testing, simple Tomcat authentication can be used, see [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webapp/WEB-INF/web.xml web.xml] for a configuration without shibboleth (cf. [https://github.com/clarin-eric/component-registry-rest/blob/master/ComponentRegistry/src/main/webapp/WEB-INF/web-shib.xml web-shib.xml]).
    100119
    101120Since a Shibboleth session can only be initiated through user interaction within the browser, all clients will need to share this browser session. For the  [[ComponentRegistryAndEditor/ReactFrontEnd|React front end]] this is trivial because it lives inside the browser and all requests go through the browser.
     121
     122There is also a configuration that disables all authentication and thus effectively puts the Component Registry in read-only mode.
    102123
    103124----
    … …  
    115136=== Testing environment ===
    116137
    117 An environment for **testing purposes** has been deployed at a VM at RZG, accessible at http://alpha-vlo.clarin.eu/component-registry-rest/rest/
    118  * get all components: [http://alpha-vlo.clarin.eu/component-registry-rest/rest/registry/components]
    119  * get all profiles: [http://alpha-vlo.clarin.eu/component-registry-rest/rest/registry/profiles]
     138An environment for **testing purposes** has been deployed (currently at a HostEurope root server), accessible via https://catalog.clarin-dev.eu/ds/ComponentRegistry
     139 * get all components: [https://catalog.clarin-dev.eu/ds/ComponentRegistry/rest/registry/components]
     140 * get all profiles: [https://catalog.clarin-dev.eu/ds/ComponentRegistry/rest/registry/profiles]
    120141
    121142Obviously, one should not rely on any data in this testing environment
    122143
    123 ==== ~~Deployment~~ ====
     144== Docker ==
    124145
    125 '''Deprecated'''
    126 {TODO: write instructions that reflect the current (post-MPI) state and process}
     146A docker image can be built that takes the compiled REST project and bundles it in an Tomcat servlet based image ready for deployment. It requires some configuration, mainly:
     147* Access context (public URL)
     148* Database URL (needs to be deployed separately)
     149* External service URLs
     150* Authentication mode
     151  * Shibboleth settings if applicable
     152* Administration users
    127153
    128 In principal, you should follow the [https://trac.mpi.nl/wiki/packagedeployment MPI deployment instructions].
     154For details, see [https://gitlab.com/CLARIN-ERIC/docker-component-registry-rest docker-component-registry-rest].
    129155
    130 Here are the step-by-step instruction "How to transfer your ComponentRegistry project to lux16" for testing. In this instruction, the example local branch directory is called "yourbranch" and  the example version, on which your work on, is called "yourversion".
    131 
    132 1) Prepare the archive of the project in yourbranch/ComponentRegistry/:
    133 
    134 mvn clean install -Plux16
    135 
    136 (This also stated in README in yourbranch/ComponentRegistry/.)  The archive should appear in yourbranch/ComponentRegistry/target/
    137 
    138 2) go to http://lux16.mpi.nl:8080/manager/html and stop "componentRegistrty". Stop the old version before you start your one.
    139 
    140 3) Now, if the archive is ready, copy it to "lux16:/lat/deployment-packages/testing/componentRegistry" . You should copy the whole archive  "ComponentRegistry-version-SNAPSHOT-mpi-deploy.tar.gz" . Make sure that you have access to lux16.
    141 
    142 4) Go to /lat/webapps/componentRegistry/.  Untar the archive there: tar -zxf /lat/deployment-packages/testing/componentRegistry/ComponentRegistry-yourversion-SNAPSHOT-mpi-deploy.tar.gz
    143 
    144 5) Go to /lat/webapps/componentRegistry/ComponentRegistry-yourversion-SNAPSHOT/war/ .  Unzip the war-file there: unzip ComponentRegistry.war
    145 
    146 6) Go to  /lat/webapps/componentRegistry/. Symbolic link "current" still  points to the old version of the project. Remove it by: rm current
    147 
    148 7) Make a new symbolic link: ln -s ComponentRegistry-yourversion-SNAPSHOT/war ./current
    149 
    150 8) Go to /lat/tomcat-webuser/conf/Catalina/localhost/ .  Now, it may happen that the xml file gets removed by TomCat, but you need it there. Use its backup: cp backup/ds#ComponentRegistry.xml .
    151 
    152 9) run log to faciliate testing: tail -f /lat/tomcat-webuser/logs/componentRegistry.log
     156Note: the REST service nor its image provide a front end, nor do they depend on it. The front end has a [https://gitlab.com/CLARIN-ERIC/docker-component-registry-front-end separate image] which can be managed together with the back end and a databae instance to provide a complete solution using a dedicated [https://gitlab.com/CLARIN-ERIC/compose_compreg Compose project].