Changeset 6410
- Timestamp:
- 07/23/15 13:14:08 (9 years ago)
- Location:
- vlo/trunk
- Files:
-
- 2 edited
-
DEPLOY-README.txt (modified) (1 diff)
-
UPGRADE.txt (modified) (2 diffs)
Legend:
- Unmodified
- Added
- Removed
-
vlo/trunk/DEPLOY-README.txt
r6298 r6410 1 VLO DEPLOY-README.txt 2 1 3 The instructions below describe the installation of the applications necessary 2 4 for browsing: the VLO web application, the Solr server, and the meta data 3 5 importer. 4 6 7 $VLO â installation folder for VLO e.g.: /opt/vlo/ 8 $CATALINA_HOME â tomcatâs installation folder 9 $SOLR_DATA - SOLR location for storing indexed data 10 xxx - VLO version number 11 5 12 1. The archive 6 13 7 The VLO importer and web application are contained in an 8 archive. In the case of version 3.0, the archive is named 14 The VLO importer and web application are contained in an archive: 9 15 10 vlo-3.0-Distribution.tar.gz16 vlo-xxx-Distribution.tar.gz 11 17 12 18 Deploying the VLO application means: 13 19 - unpacking the archive in a suitable location 14 20 - unpacking the war files contained in the archive 15 - adapt the application's context files 16 and17 - adapt the VloConfig.xml main configuration file.21 - adapt the application's context files and 22 - adapt the VloConfig.xml main configuration file and/or the dataroot config 23 included in that file 18 24 - optionally executing upgrade steps described in the UPGRADE file 19 25 20 26 2. Archive unpacking 21 27 22 Unpack the archive, for example 28 » service tomcat6 stop (or $CATALINA_HOME/bin/shutdown.sh if tomcat is 29 not installed as a service) 30 » mkdir $VLO 31 » tar -C $VLO --strip-components=1 -zxvf vlo-xxx-Distribution.tar.gz 32 33 In the tree starting in $VLO, the configuration of the application is 34 stored under subfolder config. 23 35 24 vlo-3.0-Distribution.tar.gz 36 3. Solr server & web-app installation 25 37 26 in a temporary directory: 'temp'. Next, stop the Tomcat server, e.g.: 38 » unzip -d $CATALINA_HOME/webapps/vlo $VLO/war/vlo-web-app-xxx.war 39 » unzip -d $CATALINA_HOME/webapps/vlo-solr $VLO/war/vlo-solr-xxx.war 27 40 28 /etc/init.d/tomcat6 stop 41 Modify (if needed): 42 $CATALINA_HOME/webapps/vlo/META-INF/context.xml 43 - parameter "eu.carlin.cmdi.vlo.config.location" 44 should point to VloConfig.xml, 45 e.g. value="$VLO/config/VloConfig.xml" 46 - parameter "eu.carlin.cmdi.vlo.solr.serverUrl" 47 is for custom SOLR server, leave commented for the default 48 49 $CATALINA_HOME/webapps/vlo-solr/META-INF/context.xml 50 - parameter "solr/home" 51 should point to the folder with config files for SOLR, 52 e.g. value="$VLO/config/solr"/> 53 54 and copy them to $CATALINA_HOME/conf/Catalina/localhost/ as vlo.xml and 55 vlo-solr.xml respectively. 56 » cp $CATALINA_HOME /webapps/vlo/META-INF/context.xml\ 57 $CATALINA_HOME/conf/Catalina/localhost/vlo.xml 58 » cp $CATALINA_HOME /webapps/vlo-solr/META-INF/context.xml\ 59 $CATALINA_HOME/conf/Catalina/localhost/vlo-solr.xml 60 61 Add following line to the $CATALINA_HOME/bin/setenv.sh: 62 » echo 'export JAVA_OPTS="$JAVA_OPTS -Dsolr.data.dir=$SOLR_DATA"'\ 63 >> $CATALINA_HOME/bin/setenv.sh 64 65 Optional: 66 67 Modify log4j settings in: 68 - $CATALINA_HOME/webapps/vlo/WEB-INF/classes/log4j.properties 69 - $CATALINA_HOME/webapps/vlo-solr/WEB-INF/classes/log4j.properties 70 71 Wicket runs in development mode by default, to change it add 72 following parameter to the JAVA_OPTS in 73 $CATALINA_HOME/bin/setenv.sh: 74 -Dwicket.configuration=deployment 75 or set a context parameter 'configuration' to the value 76 'deployment' in the application's context fragment. 77 78 4. Importer configuration 79 80 Modify DataRoot for importer directly in $VLO/config/VloConfig.xml or in 81 the file that is included into that file via XInclude 29 82 30 After the server has stopped, copy the contents of the vlo_parent-2.13 31 directory just created to a permanent directory, 'vlo', for example. In the 32 tree starting in vlo, the configuration of the application is stored. Also, 33 from the 83 <originName>MPI self harvest</originName> 84 <rootFile>path to the metada root folder</rootFile> 85 <prefix>http://m12404423/vlomd/</prefix> 86 <tostrip>/var/www/vlomd/</tostrip> 87 <deleteFirst>false</deleteFirst> 88 </DataRoot> 89 90 A dataRoot element describes the meta data files. The toStrip part of 91 the description is left out of the rootFile part to create a http link 92 to the metadata; the links starts with the prefix. 93 94 95 5. Importing data 96 97 importer can be found in $VLO/bin/ folder 34 98 35 vlo/bin 36 37 directory, you can run the importer application. Since this application 38 relies on the Solr server, we first need to install the Solr web application. 99 Before starting data import, first start the Tomcat server: 39 100 40 Recursively assign ownership of the entire unpacked distribution to the appropriate 41 user (i.e. 'vlouser' on catalog.clarin.eu). 101 » service tomcat6 start 42 102 43 3. Solr server installation 103 (or $CATALINA_HOME/bin/startup.sh if tomcat is not installed as a 104 service) 105 106 run importer: 44 107 45 The VLO importer and web application use Solr as their database 46 server. To install this server, assuming that the Tomcat server 47 has not started yet, copy the Solr server web application archive 48 to the Tomcat server web application directory. This could for 49 example be 50 51 /var/lib/tomcat6/webapps 52 53 Step into this directory, lets call it 'apps', 54 55 cd apps 56 cp temp/vlo-3.0/war/vlo-solr-3.0.war . 57 58 If it does not exist, create the vlo_solr directory, and unpack the web 59 application archive in it: 60 61 cd vlo_solr 62 unzip ../vlo-solr-3.0.war 63 64 After unzipping, remove the vlo-solr-3.0.war file. Next, if necessary, 65 modify the solr/home parameter in the 66 67 apps/META-INF/context.xml 68 69 file to the path where the solr server finds its configuration and stores 70 its data: 71 72 vlo/config/solr 73 74 Solr needs a location to store its index data. This needs to be configured 75 through a Java system property 'solr.data.dir'. Configure this in your 76 Tomcat instance, for example by adding the following to 77 ${catalina.home}/bin/setenv.sh: 78 79 export JAVA_OPTS="$JAVA_OPTS -Dsolr.data.dir=/var/vlo/solr/data" 80 81 The directory does not have to exist, but its parent does and the Tomcat 82 user needs write access in that location. 83 84 Copy the new context.xml file to the Tomcat configuration: 85 86 cp apps/META-INF/context.xml \ 87 $CATALINA_HOME/conf/Catalina/localhost/vlo_solr.xml 88 89 If you would like logging to be configured different from the type of 90 logging packaged, please modify 91 92 apps/WEB-INF/classes/log4j.properties 93 94 If necessary, change the ownership of the files in the tree starting in 95 96 vlo/config/solr 97 98 to that of the Tomcat user. This will enable the Solr server can store data 99 in it. Now the Solr server has been installed, the importer and web 100 application could use it. Next we will install the VLO web application. 101 102 4. Web application installation 103 104 Similar to the Solr archive, unpack 105 106 temp/vlo-3.0/war/vlo-web-app-3.0.war in 107 108 the 109 110 apps/vlo 111 112 directory. 113 114 Because the packaged configuration is suitable to very specific (development) 115 circumstances only, in most cases it does fit the production environment. You 116 can specify an alternative, external configuration file, preferably the same 117 file as the one used by the importer (see 5). Please modify 118 119 apps/META-INF/context.xml 120 121 by adding a reference to an external configuration file. By modifying this 122 file, you can adapt the VLO configuration to your needs. You might, for 123 example, need to assign another value to the solrUrl parameter also. This 124 parameter is used to let the VLO web application know where it can reach the 125 Solr server. 126 127 Instead of changing the value of the solrUrl parameter in the external 128 configuration file, you could also supply an alternative value by adding 129 the parameter to 130 131 apps/META-INF/context.xml 132 133 directly. The comments in this file will tell you how to add the parameter. 134 135 Like in the case of the Solr server, copy the apps/META-INF/context.xml 136 context file to: 137 138 /var/lib/tomcat6/conf/Catalina/localhost/vlo.xml 139 140 or to another, comparable path. 141 142 If you like, you can change the web application's way of logging. This time, 143 modifications should be applied to: 144 145 apps/WEB-INF/classes/log4j.properties 146 147 Wicket runs in development mode by default, and depending on the environment 148 you are deploying to this may be undesirable. To make sure it runs in 149 production ('deployment') mode, set the following JVM option in the 150 Tomcat JVM: 151 152 -Dwicket.configuration=deployment 153 154 OR set a context parameter 'configuration' to the value 'deployment' in the 155 application's context fragment. 156 157 Now the web application has been installed and configured, we only need to 158 have a look at the VLO importer's configuration. 159 160 5. Importer configuration 161 162 As mentioned in the description of the web application installation, a 163 typical setup needs an external configuration file. For consistency reasons 164 both the web application and application that imports data into the Solr 165 data base should use one and the same configuration file. So if, for example, 166 the web application's configuration is in 167 168 $VLO_CONFIG/VloConfig.xml, use 169 170 ./vlo_solr_importer.sh $VLO_CONFIG/VloConfig.xml 171 172 to run the importer in 'bin'. A default configuration file is supplied as 173 'config/VloConfig.xml'. The importer script will default to this location 174 if no location is specified on the command line. 175 176 Most likely, in the configuration file, the dataRoot values need to be 177 changed. This is what a dataRoot definition could look like: 178 179 <originName>MPI self harvest</originName> 180 <rootFile>/var/www/vlomd/self/</rootFile> 181 <prefix>http://m12404423/vlomd/</prefix> 182 <tostrip>/var/www/vlomd/</tostrip> 183 <deleteFirst>false</deleteFirst> 184 </DataRoot> 185 186 A dataRoot element describes the meta data files. The toStrip part of the 187 description is left out of the rootFile part to create a http link to the 188 meta data; the links starts with the prefix. 189 190 Apart from the dataRoot values, the solrUrl parameter might need to be 191 changed. Please note that the context path value defined in 192 193 $CATALINA_HOME/conf/Catalina/localhost/vlo_solr.xml 194 195 or in a file equivalent to this one, should be reflected in the value of 196 this parameter. 197 198 Whenever the definition of the database changes, it is recommended to clear 199 the database before starting an importer run. To do this, make sure that the 200 'deletaAllFirst' parameter equals true before the run. After that, the 201 original value of the parameter could be restored. 202 203 6. Importing data 204 205 Before starting data import, first start the Tomcat server, e.g.: 206 207 /etc/init.d/tomcat6 start 208 209 To be sure the Solr server is working as it should, inspect the Tomcat log 210 file directory, and in that directory, check the solr.log and vlo.log files. 211 212 Next, you can run the importer by starting the 213 214 ./vlo_solr_importer.sh 215 216 script in the vlo/bin directory. Messages are logged to the a log file. 217 Because meta data is not static, it is recommended to run the importer a 218 couple of times a week. Please note that, given the current (04.2014) set of 219 data, a run typically takes between two and four hours. 220 221 More information about running the importer can be found in the README.txt 222 file. 223 108 » $VLO/bin/vlo_solr_importer.sh 109 110 (optionally pass the path to custom VloConfig.xml, default is in 111 $VLO/config) 112 113 The importer logs information in $VLO/log 114 115 Because meta data is not static, it is recommended to run the importer a 116 couple of times a week. Please note that, for the tar file with metadata 117 of 270 MB run approximately takes 2h. -
vlo/trunk/UPGRADE.txt
r6390 r6410 15 15 appropriate user, i.e. 'vlouser' on catalog.clarin.eu) 16 16 17 - In the 'config' directory, adjust VloConfig.xml as needed (make a backup 18 first). In particular, make sure that the following instance specific sections 19 are kept from the current version: 20 - dataRoots and its children 21 - solrUrl 22 - homeUrl 23 - deleteAllFirst 24 - maxDaysInSolr 25 - Make any further version specific adjustments as described in this document 17 - In the 'config' directory, inspect VloConfig.xml and the data roots 18 configuration file that is included into this file by means of XInclude. It is 19 advised to perform a diff between the old and new versions of these files. 20 Depending on configuration preferences, you may want to copy settings from the 21 old configuration to the new one. 26 22 27 23 - Stop the Tomcat or at least the VLO web app and Solr applications … … 32 28 ./vlo_solr_importer.sh -c ../config/VloConfig.xml 33 29 34 --------------------------- 35 Upgrading from 3.2 to 3.2.1 36 --------------------------- 30 -------------------------- 31 Upgrading from 3.2 to 3.3 32 -------------------------- 33 * Take note of the change in VloConfig.xml which moves the data roots definition 34 out of VloConfig.xml by means of XInclude. The configuration file now gets 35 prepared for deployment environments at build time, so it is no longer necessary 36 to copy over old configuration values in all cases. 37 37 38 * In the context fragment for the VLO web application, ADD the following 38 39 attribute to the <Context> root element:
Note: See TracChangeset
for help on using the changeset viewer.
