= CMDI 1.2 Rollout = [[PageOutline]] To take place on '''7 and 8 July 2016'''. Also see the previously published [https://office.clarin.eu/v/CE-2015-0702-CMDI_12_rollout_plan.pdf rollout plan] (which is slightly outdated in the details by now). == Rough schedule == * Thursday * ~10:30~12:00 Component Registry preparation + beta * ~13:00~15:00 Component Registry production * ~15:00~17:00 VLO beta * Friday * ~9:00~12:00 VLO production * ~13:00~17:00 New harvest, testing and announcements == Detailed release workflow == Steps in order of execution === Preparation === Preparation steps can take place a couple of days before the deployment * [x] '''Twan''': Announce downtime on 7/8 July * [x] '''Twan''': Prepare [#Componentconversion component conversion], i.e. fix/remove problematic components and profiles * [x] '''!Willem/Menzo''': Fix specifications with implicit cardinalities * [x] '''Menzo''': Make deployment packages for the harvester (ultimately 6 July) * [https://datashare.rzg.mpg.de/s/ADQdfkhiNjPJ8Gm/download oai-harvest-manager-0.5.a06cc3.tar.gz] (direct link) * [https://raw.githubusercontent.com/clarin-eric/oai-harvest-manager/master/src/main/scripts/oaiharvest.sh oaiharvest.sh] (updated handling of log files, commented out Meertens sync as Meertens is (for now) part of the main CLARIN harvest) * [https://raw.githubusercontent.com/clarin-eric/oai-harvest-manager/master/src/main/scripts/oaiharvest-env.sh oaiharvest-env.sh] (environment vars) * cron job for Meertens harvest can be disabled (for now) * [x] '''Menzo''': Run 'Others' harvest on alpha, package results for deployment * [https://datashare.rzg.mpg.de/index.php/s/k29STUM444lCqeO/download others.tar.bz2] (direct link; to replace the dirs in /srv/vlo-data/others/) * [https://datashare.rzg.mpg.de/index.php/s/IkMj8RDqwaK60pd/download others-sil.tar.bz2] (direct link; to be added to /srv/vlo-data/others/) * [x] '''Menzo''': Run 'Clarin' harvest on alpha, package results for deployment * [https://datashare.rzg.mpg.de/s/r2iYz9U41AxWQQv/download clarin.tar.bz2] (direct link; to replace the dirs in /srv/vlo-data/clarin/) * [x] '''Twan''': Make deployment packages for the VLO (ultimately 6 July) * [https://clarin.fz-juelich.de/owncloud/public.php?service=files&t=a9bc18044677f2afe6ab0d56b6916877&download vlo-4.0.0-beta2-Distribution.tar.gz] (direct link) * [https://clarin.fz-juelich.de/owncloud/public.php?service=files&t=1808294b0438dcd65517a5ff0ff7fc35&download vlo-4.0.0-Distribution.tar.gz] (direct link) ==== Toolkit ==== One day before the deployment * [x] '''Menzo''': Change [https://github.com/clarin-eric/cmdi-toolkit/blob/master/src/main/resources/toolkit/upgrade/cmd-record-1_1-to-1_2.xsl#L17 the default Component Registry location] from the alpha server to the production server * [x] '''Menzo''': Tag/release toolkit on !GitHub, make development branch (master should be stable) * https://github.com/clarin-eric/cmdi-toolkit/releases/tag/cmdi-1.2.0 * [x] '''Menzo''': Check if the toolkit is available and up-to-date at [https://infra.clarin.eu/CMDI/1.x] * [x] '''Twan''': Make a runnable package with ready to use configuration file for the conversion tool * [https://clarin.fz-juelich.de/owncloud/public.php?service=files&t=d9a1ff1c2455dd526114bf2408acf0fb&download component-updater-2.1.0.zip] (direct link) * [x] '''Twan''': Make a deployment package for the Component Registry * [https://clarin.fz-juelich.de/owncloud/public.php?service=files&t=bd8620cf3f278ea98b5dd7e769f47a0a&download component-registry-rest-2.1.0-beta-mpi-deploy.tar.gz] (direct link - '''beta''') * [https://clarin.fz-juelich.de/owncloud/public.php?service=files&t=6eead8fd11938b96fb9d5bfaa5b0577d&download component-registry-rest-2.1.0-mpi-deploy.tar.gz] (direct link) === Component Registry === Preparation and beta deployment * [x] '''Willem''': Put the '''production''' Component Registry in read-only mode * [x] '''Willem''': Send a dump of the '''production''' database to Menzo: {{{beta-vlo-clarin:/home/menwin/compreg.pre.cmdi1.2.rollout.sql.bz2}}} * [x] '''Menzo''': Check for newly defined implicit cardinalties and fix if needed (Menzo can generate a `fix.sql` file on basis of the dump) * [https://datashare.rzg.mpg.de/index.php/s/6I1dWWqX0GvjJwR/download compreg.pre.cmdi1.2.rollout.fix.sql] (direct link) * [x] '''Willem''': Apply the database fix script (if needed) in '''production''' * [x] '''Willem''': Make a new dump of the production database and transfer this to the Component Registry '''beta''' host (`dev-sp.clarin.eu`) + send a copy to Twan * [x] '''Twan''': Run a 'dry' [#Componentconversion component conversion] on a dump of the Component Registry database to see if there are no new problematic cases * [x] '''Willem''': Replace the database on the Component Registry '''beta''' host with the fresh dump * [x] '''Willem''': Run the upgrade tool (see below for details) on '''beta''' and make sure that it terminates without any errors * in case of errors, most likely one or more components need to be fixed * [x] '''Willem''': Deploy and start the Component Registry version with support for CMDI 1.2 on '''beta''' Production update and deployment: * [x] '''Willem''': Stop the '''production''' Component Registry * [#Componentconversion Convert] of all component and profile definitions within the '''production''' Component Registry with the component upgrade tool: * [x] '''Willem''': Run the upgrade tool (see below for details) and make sure that it terminates without any errors * in case of errors, most likely one or more components need to be fixed * [x] '''Willem''': Make a backup of the freshly converted database * [x] '''Willem''': Deploy and start the Component Registry version with support for CMDI 1.2 on '''production''' === VLO and harvester === On beta: * [x] '''Willem''': Deploy the VLO on beta * [x] '''Willem''': Put the prepared harvest results on beta * [x] '''Willem''': (re)move /srv/vlo-data/clarin/* * [x] '''Willem''': cd /srv/vlo-data/clarin/; tar xfj clarin.tar.bz2 * [x] '''Willem''': cd /srv/vlo-data/clarin/results; mv cmdi cmdi-1_1; mv cmdi-1_2 cmdi * [x] '''Willem''': (re)move /srv/vlo-data/others/* * [x] '''Willem''': cd /srv/vlo-data/others/; tar xfj others.tar.bz2 * [x] '''Willem''': cd /srv/vlo-data/others/; tar xfj others-sil.tar.bz2 (should merge with previously created dirs) * [x] '''Willem''': cd /srv/vlo-data/others/results; mv cmdi cmdi-1_1; mv cmdi-1_2 cmdi * [x] '''Willem''': Run an import on beta On production: * [x] '''Willem''': Transfer the Solr index from beta and the prepared harvest results to production in order to 'bootstrap' the VLO * [x] '''Willem''': cd /srv/vlo-data/work/vu/results; mv cmdi cmdi-1_1; mv cmdi-1_2 cmdi * [x] '''Willem''': Deploy the OAI harvester with support for CMDI 1.2, configure harvest result to CMDI 1.2 * [x] '''Willem''': Deploy the VLO version with support for CMDI 1.2 * [x] '''Willem''': Disable the Meertens harvest cron job (for now it's included again in the main CLARIN harvest) === Testing === * [x] '''Willem''': Start a VLO import manually * [x] '''Willem''': Run a harvest with conversion to CMDI 1.2 (manually or depend on cronjob) * [x] '''Willem''': Run a new VLO import (manually or depend on cronjob) Next day: * [x] '''Menzo''': Validate harvest results (CMDI 1.1 and CMDI 1.2) * [https://datashare.rzg.mpg.de/s/klLDBYMkiGBFHL1/download catalog-cmdi-1_1-validation-20160705.log] (direct link; CMDI 1.1 validation of the 4 July CLARIN harvest against the CR **before** the update) \\ ```validation result: 18.27% failure rate (files: 760387 total, 621496 passed, 138891 failed; 12.03 GB total, 1410 files/second, 22.85 MB/second)``` * [https://datashare.rzg.mpg.de/s/hcUzv1osurrKIWy/download alpha-cmdi-1_1-validation-20160705.log] (direct link; CMDI 1.1 validation of the 4 July CLARIN harvest against the CR on the alpha server) \\ ```validation result: 18.27% failure rate (files: 760387 total, 621496 passed, 138891 failed; 12.03 GB total, 1206 files/second, 19.55 MB/second)``` * [https://datashare.rzg.mpg.de/s/mSgagZQm4UnfVWy/download catalog-cmdi-1_1-validation-20160706.log] (direct link; CMDI 1.1 validation of the 4 July CLARIN harvest against the CR **after** the update) \\ ```validation result: 18.27% failure rate (files: 760387 total, 621496 passed, 138891 failed; 12.03 GB total, 812 files/second, 13.16 MB/second)``` * [https://www.evernote.com/l/AG6ID60zEGJIyryYHKuzwOQ0l1CUD_0Wm1g analysis of the differences between 1.1 and 1.2 validation] * [x] '''Twan''': Validate import results === Announcements and documentation === * [x] Make a draft version of the specification available as PDF * [https://datashare.rzg.mpg.de/s/S7WJjEDraLdc9Ai CMDI-1_2-specification-draft-20160706.pdf] * [x] Make an update version of the executive summary available * [x] Update [http://clarin.eu/cmdi CMDI and ][http://clarin.eu/cmdi12 CMDI 1.2 pages of the clarin.eu website] * [x] Make a news item/blog post about CMDI 1.2 pointing to sources of information and laying out the roadmap for further CMDI 1.2 infrastructure integration == Component conversion == A utility called the [https://github.com/clarin-eric/component-registry-rest/tree/develop/ComponentUpdater ComponentUpdater] applies the upgrade stylesheet to all components in the Component Registry database and validates the output. If all components transform and validate successfully, it commits the changes. A prepared (compiled and pre-configured) package will be made available to run on production at the time of rollout of CMDI 1.2. === Problematic components === The components (and profiles) listed by id below become invalid (with respect to schema or schematron rules) after conversion. They need to be adapted or removed before the migration. '''Status''': all changes applied to production database on 2016/07/13. Original components attached as retrieved on that day before applying changes. ==== Components that need to be fixed ==== - [x] clarin.eu:cr1:c_1280305685305 - minimal cardinality > maximum cardinality - [x] clarin.eu:cr1:c_1311927752346 - duplicate (empty) enumeration items ==== Components that can be removed ==== These are all either unused components clearly created for test purposes or have been deleted by the user. - [x] clarin.eu:cr1:p_1337778924929 - [x] clarin.eu:cr1:p_1288172614022 - [x] clarin.eu:cr1:p_1288172614024 - [x] clarin.eu:cr1:p_1290431694477 - [x] clarin.eu:cr1:p_1274880881900 - [x] clarin.eu:cr1:p_1317737532981 - [x] clarin.eu:cr1:c_1311927752348