Changeset 6267


Ignore:
Timestamp:
06/04/15 07:57:04 (9 years ago)
Author:
Twan Goosen
Message:

made branch up to date with trunk

Location:
vlo/branches/vlo-ticket761
Files:
62 edited
12 copied

Legend:

Unmodified
Added
Removed

  • vlo/branches/vlo-ticket761

  • vlo/branches/vlo-ticket761/CHANGES.txt

    r6160 r6267  
    11Bug Fixes and Features
    22
     3VLO 3.2 (June 2015)
     4       
     5Mapping:
     6* Added search facet "Availability" based on licensing information
     7        <https://trac.clarin.eu/ticket/393>
     8* Non-English language names are ignored (if language code is not used)
     9        <https://trac.clarin.eu/ticket/170>
     10* Mapping of language name variants to canonical names (OLAC)
     11        <https://trac.clarin.eu/ticket/749>
     12* Mapping of language tags without olac-language attribute (OLAC)
     13        <https://trac.clarin.eu/ticket/412>
     14
     15Search interface:
     16* Added tooltips to facets with facet descriptions
     17        <https://trac.clarin.eu/ticket/711>
     18* Search term can be removed without removing facet selection in 'bread crumbs'
     19        <https://trac.clarin.eu/ticket/619>
     20* Search result ranking by relevance
     21        <https://trac.clarin.eu/ticket/575>
     22               
     23Record page:
     24* Paging of resources in record page
     25        <https://trac.clarin.eu/ticket/565>
     26* Improved handle resolution for resources in record page
     27        <https://trac.clarin.eu/ticket/550>
     28* Fixed handle link in record page for cases that contain an '@' character
     29        <https://trac.clarin.eu/ticket/625>
     30* Link to language info page uses new URL scheme
     31        <https://trac.clarin.eu/ticket/753>
     32
     33Web application general:
     34* VLO web pages contain a canonical URL link (improving search engine behaviour)
     35        <https://trac.clarin.eu/ticket/135>
     36       
    337VLO 3.1 (March 2015)
    438* Added the 'license' facet

  • vlo/branches/vlo-ticket761/DEPLOY-README.txt

    r6102 r6267  
    2424         vlo-3.0-Distribution.tar.gz
    2525
    26    in a temporary directory: 'temp'. Next, stop the Tomcat server:
    27 
    28         /etc/init.d/tomcat6 stop
     26   in a temporary directory: 'temp'. Next, stop the Tomcat server, e.g.:
     27
     28     /etc/init.d/tomcat6 stop
    2929
    3030   After the server has stopped, copy the contents of the vlo_parent-2.13
    … …  
    1931936. Importing data
    194194
    195    Before starting data import, first start the Tomcat server:
    196 
    197         /etc/init.d/tomcat6 start
     195   Before starting data import, first start the Tomcat server, e.g.:
     196
     197         /etc/init.d/tomcat6 start
    198198
    199199   To be sure the Solr server is working as it should, inspect the Tomcat log
    … …  
    202202   Next, you can run the importer by starting the
    203203
    204         ./vlo_solr_importer.sh
    205 
    206    script in the vlo/bin directory. Messages are logged to the console. Because
    207    meta data is not static, it is recommended to run the importer a couple of
    208    times a week. Please note that, given the current (04.2014) set of data, a 
    209    run typically takes between two and four hours.
     204         ./vlo_solr_importer.sh
     205
     206   script in the vlo/bin directory. Messages are logged to the a log file.
     207   Because meta data is not static, it is recommended to run the importer a
     208   couple of times a week. Please note that, given the current (04.2014) set of
     209   data, a  run typically takes between two and four hours.
     210   
     211   More information about running the importer can be found in the README.txt
     212   file.
     213   

  • vlo/branches/vlo-ticket761/README.txt

    r6101 r6267  
     1= Table of Contents =
     2- What is the VLO?
     3- What should I read?
     4- Setting up
     5        - Preparing for release
     6- Using the VLO
     7        - Running an import
     8       
    19= What is the VLO? =
    210
    … …  
    2634Some general development notes.
    2735
    28 == Setting up ==
     36= Setting up =
    2937
    3038Follow the instructions in DEPLOY-README to set up a development environment.
    … …  
    8290files.
    8391
     92= Using the VLO =
     93
     94After a successful deployment and configuration, you should be able to browse to
     95the VLO web app and browse the imported records.
     96
     97If the VLO is empty, you will need to run an import first.
     98
     99== Running an import ==
     100
     101To run an import, go the 'bin' child directory of the VLO application directory
     102and run
     103
     104        ./vlo_solr_importer.sh
     105       
     106as the appropriate user (e.g. 'vlouser').
     107
     108It's advisable to run this in a detached background process (for example using
     109"screen") because an import can take quite a long time depending on the amount
     110of records to be imported.
     111
     112Some progress information is logged to a file 'log/vlo-importer.log'. It gets
     113rotated automatically by the import process.
     114
     115For a fresh import, even when the VLO is not configured to delete all documents
     116from the index first, you can simply remove the contents of the Solr data
     117directory (check server configuration documentation for the exact location).

  • vlo/branches/vlo-ticket761/UPGRADE.txt

    r6160 r6267  
    1313appropriate user (i.e. 'vlouser' on catalog.clarin.eu)
    1414
    15 - In the 'config' directory, adjust VloConfig.xml as needed (make a backup first). In
    16         particular, make sure that the following instance specific sections are kept:
     15- In the 'config' directory, adjust VloConfig.xml as needed (make a backup
     16first). In particular, make sure that the following instance specific sections
     17are kept from the current version:
    1718        - dataRoots and its children
    1819        - solrUrl
    … …  
    2526- Replace the 'current' symlink with a link to location of the new VLO directory
    2627
    27 - If needed, flush the Solr index (by removing the data directory)
     28- If needed, flush the Solr index (by removing the data directory's content)
    2829- In 'bin', run the importer using the updated configuration file:
    2930    ./vlo_solr_importer.sh -c ../config/VloConfig.xml
     31
     32---------------------------
     33Upgrading from 3.1 to 3.2
     34---------------------------
     35* Take note of the following changes in VloConfig.xml:
     36
     37- 'facetConceptsFile' now defines the location of the facet/concept mapping
     38definition on disk; to use the default bundled definition, leave it empty.
     39Before this used to have the value '/facetConcepts.xml' which will no longer be
     40valid - it can safely be replaced with either the empty string or
     41'facetConcepts.xml' to use the file in the same directory as VloConfig.xml.
     42
     43- 'languageLinkPrefix' element has been REPLACED by 'languageLinkTemplate';
     44remove the former and add the following to existing configurations:
     45
     46        <languageLinkTemplate>https://infra.clarin.eu/content/language_info/data/{}.html</languageLinkTemplate>
     47
     48- 'languageNameVariantsUrl' and 'licenseAvailabilityMapUrl' are now required
     49mapping configuration elements. So the following should normally be added:
     50
     51    <languageNameVariantsUrl>/LanguageNameVariantsMap.xml</languageNameVariantsUrl>   
     52    <licenseAvailabilityMapUrl>/LicenseAvailabilityMap.xml</licenseAvailabilityMapUrl>
     53
     54- optional 'showResultScores' element, defaults to false. Should be false except
     55when debugging result ranking.
     56
     57- The facet field 'availability' is now available
     58
     59* Be aware that the default maximum heap space of the importer has been
     60increased from 2GB to 3GB (by means of a change in bin/vlo_solr_importer.sh).
    3061
    3162---------------------------

  • vlo/branches/vlo-ticket761/pom.xml

    r5991 r6267  
    1111         in each module's pom.xml
    1212    -->
    13     <version>3.2-SNAPSHOT</version>
     13    <version>3.3-SNAPSHOT</version>
    1414   
    1515    <description>Virtual Language Observatory (VLO) parent project</description>
    … …  
    2828
    2929    <properties>
    30         <solr.version>4.8.1</solr.version>
    31         <slf4j.version>1.7.10</slf4j.version>
     30        <solr.version>4.10.4</solr.version>
     31        <slf4j.version>1.7.12</slf4j.version>
    3232        <saxon.version>9.5.1-8</saxon.version>
    3333        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

  • vlo/branches/vlo-ticket761/vlo-commons/pom.xml

    r5991 r6267  
    66        <groupId>eu.clarin.cmdi</groupId>
    77        <artifactId>vlo</artifactId>
    8         <version>3.2-SNAPSHOT</version>
     8        <version>3.3-SNAPSHOT</version>
    99    </parent>
    1010    <groupId>eu.clarin.cmdi</groupId>
    1111    <artifactId>vlo-commons</artifactId>
    12     <version>3.2-SNAPSHOT</version>
     12    <version>3.3-SNAPSHOT</version>
    1313    <name>VLO Commons</name>
    1414    <url>http://maven.apache.org</url>

  • vlo/branches/vlo-ticket761/vlo-commons/src/main/java/eu/clarin/cmdi/vlo/FacetConstants.java

    r6188 r6267  
    2727    public static final String FIELD_NATIONAL_PROJECT = "nationalProject";
    2828    public static final String FIELD_KEYWORDS = "keywords";
     29    /**
     30     * Solr pseudo-field that reveals the ranking score
     31     *
     32     * @see
     33     * https://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_see_the_relevancy_scores_for_search_results
     34     */
     35    public static final String FIELD_SOLR_SCORE = "score";
    2936
    3037    //The _ facets are not meant to be shown to users.
    … …  
    3542    public static final String FIELD_SEARCH_SERVICE = "_contentSearchRef";
    3643    public static final String FIELD_LAST_SEEN = "_lastSeen";
     44    public static final String FIELD_HIERARCHY_WEIGHT = "_hierarchyWeight";
     45    public static final String FIELD_IS_PART_OF = "_isPartOf";
     46    public static final String FIELD_HAS_PART = "_hasPart";
    3747
    3848    /**

  • vlo/branches/vlo-ticket761/vlo-commons/src/main/java/eu/clarin/cmdi/vlo/StringUtils.java

    r4480 r6267  
    11package eu.clarin.cmdi.vlo;
     2
     3import java.util.Arrays;
     4import java.util.HashSet;
     5import java.util.Set;
    26
    37public final class StringUtils {
    … …  
    4347        return result;
    4448    }
     49   
     50    private static final Set<Character> reservedCharacters = new HashSet<Character>(Arrays.asList('!', '*', '\'', '(',
     51            ')', ';', ':', '@', '&', '=', '+', '$', ',', '/', '?', '#', '[', ']'));
     52       
     53    /**
     54     * Return normalized String where all reserved characters in URL encoding
     55     * are replaced by their ASCII code (in underscores)
     56     *
     57     * @param idString String that will be normalized
     58     * @return normalized version of value where all reserved
     59     * characters in URL encoding are replaced by their ASCII code
     60     */
     61    public static String normalizeIdString(String idString) {
     62        StringBuilder normalizedString = new StringBuilder();
     63        for (int i = 0; i < idString.length(); i++) {
     64            Character character = idString.charAt(i);
     65            if (reservedCharacters.contains(character)) {
     66                normalizedString.append("_").append((int) character).append("_");
     67            } else {
     68                normalizedString.append(character);
     69            }
     70        }
     71        return normalizedString.toString();
     72    }
    4573
    4674}

  • vlo/branches/vlo-ticket761/vlo-commons/src/main/java/eu/clarin/cmdi/vlo/config/AbstractXmlVloConfigFactory.java

    r4517 r6267  
    1919import java.io.IOException;
    2020import java.io.InputStream;
     21import java.net.URI;
    2122import javax.xml.bind.JAXBException;
    2223import javax.xml.transform.stream.StreamSource;
    … …  
    4142        final InputStream fileStream = getXmlConfigurationInputStream();
    4243        try {
    43             return marshaller.unmarshal(new StreamSource(fileStream));
     44            final VloConfig config = marshaller.unmarshal(new StreamSource(fileStream));
     45            config.setConfigLocation(getLocation());
     46            return config;
    4447        } catch (JAXBException ex) {
    4548            throw new RuntimeException("Could not deserialize configuration file", ex);
    … …  
    5659     */
    5760    protected abstract InputStream getXmlConfigurationInputStream() throws IOException;
     61   
     62    protected abstract URI getLocation();
    5863
    5964}

  • vlo/branches/vlo-ticket761/vlo-commons/src/main/java/eu/clarin/cmdi/vlo/config/DefaultVloConfigFactory.java

    r4517 r6267  
    1818
    1919import java.io.InputStream;
     20import java.net.URI;
     21import java.net.URISyntaxException;
    2022
    2123/**
    … …  
    3739        return getClass().getResourceAsStream(DEFAULT_CONFIG_RESOURCE);
    3840    }
     41
     42    @Override
     43    protected URI getLocation() {
     44        try {
     45            return getClass().getResource(DEFAULT_CONFIG_RESOURCE).toURI();
     46        } catch (URISyntaxException ex) {
     47            throw new RuntimeException("Invalid config file URI", ex);
     48        }
     49    }
     50
    3951}

  • vlo/branches/vlo-ticket761/vlo-commons/src/main/java/eu/clarin/cmdi/vlo/config/VloConfig.java

    r6160 r6267  
    11package eu.clarin.cmdi.vlo.config;
    22
    3 import java.io.UnsupportedEncodingException;
    4 import java.net.URLEncoder;
     3import java.net.URI;
    54import java.util.ArrayList;
    65import java.util.Collection;
    … …  
    1918public class VloConfig {
    2019
     20    public static final String LANGUAGE_LINK_TEMPLATE_LANGUAGE_CODE_PLACEHOLDER = "{}";
     21    public static final String DEFAULT_FACET_CONCEPTS_RESOURCE_FILE = "/facetConcepts.xml";
     22
    2123    /*
    2224     * VLO application parameter members<br><br>
    … …  
    6466    //(required = false)
    6567    private String nationalProjectMapping = "";
    66    
     68
    6769    private String organisationNamesUrl;
    68    
     70
    6971    private String languageNameVariantsUrl;
    70    
     72
    7173    private String licenseAvailabilityMapUrl;
    7274
    … …  
    9496    private String imdiBrowserUrl = "";
    9597
    96     private String languageLinkPrefix = "";
     98    private String languageLinkTemplate = "";
    9799
    98100    // web application user interface
    … …  
    104106
    105107    private String feedbackFromUrl = "";
     108
     109    private boolean showResultScores = false;
     110
    106111    /**
    107112     * A set of fields to be excluded from display<br><br>
    … …  
    141146
    142147    private String collectionFacet;
    143    
     148
    144149    // test related parameters
    145150    //(required = false)
    … …  
    152157    private String cqlEndpointAlternative = "";
    153158
     159    private URI configLocation;
     160
    154161    /**
    155162     * Get and set methods for web application parameter members<br><br>
    … …  
    596603     *
    597604     * @return the value
    598      */
    599     public String getLanguageLinkPrefix() {
    600         return languageLinkPrefix;
     605     * @see #LANGUAGE_LINK_TEMPLATE_LANGUAGE_CODE_PLACEHOLDER
     606     */
     607    public String getLanguageLinkTemplate() {
     608        return languageLinkTemplate;
    601609    }
    602610
    … …  
    608616     *
    609617     * @param param the value
    610      */
    611     public void setLanguageLinkPrefix(String param) {
    612         languageLinkPrefix = param;
     618     * @see #LANGUAGE_LINK_TEMPLATE_LANGUAGE_CODE_PLACEHOLDER
     619     */
     620    public void setLanguageLinkTemplate(String param) {
     621        languageLinkTemplate = param;
    613622    }
    614623
    … …  
    943952        reverseProxyPrefix = param;
    944953    }
    945    
    946         /**
     954
     955    /**
    947956     * Get the value of the organisationNamesUrl parameter<br><br>
    948957     *
    … …  
    967976        organisationNamesUrl = param;
    968977    }
    969    
     978
    970979    /**
    971980     * Get the value of the languageNameVariantsUrl parameter<br><br>
    … …  
    9911000        languageNameVariantsUrl = param;
    9921001    }
    993    
    994    /**
     1002
     1003    /**
    9951004     * Get the value of the licenseAvailabilityMapUrl parameter<br><br>
    9961005     *
    … …  
    10631072        cqlEndpointAlternative = param;
    10641073    }
     1074
     1075    /**
     1076     * Sets the originating file for the current configuration
     1077     *
     1078     * @param configFile the file this config was read from
     1079     */
     1080    public void setConfigLocation(URI configFile) {
     1081        this.configLocation = configFile;
     1082    }
     1083
     1084    /**
     1085     *
     1086     * @return the file this config was read from. may be null!
     1087     */
     1088    public URI getConfigLocation() {
     1089        return configLocation;
     1090    }
     1091
     1092    public void setShowResultScores(boolean showResultScores) {
     1093        this.showResultScores = showResultScores;
     1094    }
     1095
     1096    public boolean isShowResultScores() {
     1097        return showResultScores;
     1098    }
    10651099}

  • vlo/branches/vlo-ticket761/vlo-commons/src/main/java/eu/clarin/cmdi/vlo/config/XmlVloConfigFactory.java

    r4520 r6267  
    1919import java.io.IOException;
    2020import java.io.InputStream;
     21import java.net.URI;
     22import java.net.URISyntaxException;
    2123import java.net.URL;
    2224
    … …  
    3133
    3234    /**
    33      * 
     35     *
    3436     * @param configLocation URL from which the configuration should be read
    3537     */
    … …  
    4143        return configLocation.openStream();
    4244    }
     45
     46    @Override
     47    protected URI getLocation() {
     48        try {
     49            return configLocation.toURI();
     50        } catch (URISyntaxException ex) {
     51            throw new RuntimeException("Invalid config file URI", ex);
     52        }
     53    }
     54
    4355}

  • vlo/branches/vlo-ticket761/vlo-commons/src/main/resources/VloConfig.xml

    r6160 r6267  
    4747    <helpUrl>http://www.clarin.eu/vlo</helpUrl>
    4848   
    49     <facetConceptsFile>/facetConcepts.xml</facetConceptsFile>
     49    <!-- leave empty to use default packaged definition -->
     50    <facetConceptsFile><!-- use default packaged definition --></facetConceptsFile>
    5051   
    5152    <profileSchemaUrl>http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/profiles/{PROFILE_ID}/xsd</profileSchemaUrl>
    … …  
    139140    <countryComponentUrl>http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/components/clarin.eu:cr1:c_1271859438104/xml</countryComponentUrl>
    140141   
    141     <languageLinkPrefix>http://infra.clarin.eu/service/language/info.php?code=</languageLinkPrefix>
     142    <languageLinkTemplate>https://infra.clarin.eu/content/language_info/data/{}.html</languageLinkTemplate>
    142143   
    143144    <feedbackFromUrl>http://www.clarin.eu/node/3759?url=</feedbackFromUrl>
    … …  
    156157   
    157158    <cqlEndpointAlternative>http://cqlservlet.mpi.nl/</cqlEndpointAlternative>
     159   
     160    <!-- makes the score for the result ranking visible in the search results -->
     161    <showResultScores>false</showResultScores>
    158162</VloConfig>

  • vlo/branches/vlo-ticket761/vlo-commons/src/test/java/eu/clarin/cmdi/vlo/config/DefaultVloConfigFactoryTest.java

    r6160 r6267  
    22
    33import java.io.File;
    4 import java.io.UnsupportedEncodingException;
    5 import java.net.URLEncoder;
    6 import java.util.ArrayList;
    74import java.util.Arrays;
    8 import java.util.HashSet;
    95import java.util.List;
    106import java.util.Set;
    … …  
    499495
    500496    /**
    501      * Test the getLanguageLinkPrefix method
     497     * Test the getLanguageLinkTemplate method
    502498     */
    503499    @Test
    … …  
    506502        System.out.println("getLanguageLinkPrefix");
    507503
    508         String expResult = "http://infra.clarin.eu/service/language/info.php?code=";
    509         String result = config.getLanguageLinkPrefix();
     504        String expResult = "https://infra.clarin.eu/content/language_info/data/{}.html";
     505        String result = config.getLanguageLinkTemplate();
    510506
    511507        assertEquals(expResult, result);
    … …  
    520516        System.out.println("setLanguageLinkPrefix");
    521517
    522         String param = "http://infra.clarin.eu/service/language/info.php?code=";
    523 
    524         config.setLanguageLinkPrefix(param);
    525 
    526         String result = config.getLanguageLinkPrefix();
     518        String param = "https://infra.clarin.eu/content/language_info/data/{}.html";
     519
     520        config.setLanguageLinkTemplate(param);
     521
     522        String result = config.getLanguageLinkTemplate();
    527523
    528524        assertEquals(param, result);

  • vlo/branches/vlo-ticket761/vlo-distribution/pom.xml

    r5991 r6267  
    77        <groupId>eu.clarin.cmdi</groupId>
    88        <artifactId>vlo</artifactId>
    9         <version>3.2-SNAPSHOT</version>
     9        <version>3.3-SNAPSHOT</version>
    1010    </parent>
    1111   
    … …  
    1414    <artifactId>vlo-distribution</artifactId>
    1515    <packaging>pom</packaging>
    16     <version>3.2-SNAPSHOT</version>
     16    <version>3.3-SNAPSHOT</version>
    1717    <name>VLO Distribution</name>
    1818    <description>

  • vlo/branches/vlo-ticket761/vlo-distribution/src/main/assembly/installPackageBuilder.xml

    r6101 r6267  
    7575            <excludes>
    7676                <exclude>VloConfig.xml</exclude>
     77                <exclude>facetConcepts.xml</exclude>
    7778            </excludes>
    7879            <outputDirectory>bin</outputDirectory>
    … …  
    9394            <includes>
    9495                <include>VloConfig.xml</include>
     96                 <include>facetConcepts.xml</include>
    9597            </includes>
    9698            <outputDirectory>config</outputDirectory>

  • vlo/branches/vlo-ticket761/vlo-importer/pom.xml

    r5991 r6267  
    77        <groupId>eu.clarin.cmdi</groupId>
    88        <artifactId>vlo</artifactId>
    9         <version>3.2-SNAPSHOT</version>
     9        <version>3.3-SNAPSHOT</version>
    1010    </parent>
    1111   
    … …  
    1313    <groupId>eu.clarin.cmdi</groupId>
    1414    <artifactId>vlo-importer</artifactId>
    15     <version>3.2-SNAPSHOT</version>
     15    <version>3.3-SNAPSHOT</version>
    1616    <packaging>jar</packaging>
    1717
    … …  
    178178            <groupId>commons-cli</groupId>
    179179            <artifactId>commons-cli</artifactId>
    180             <version>1.2</version>
     180            <version>1.3</version>
    181181            <type>jar</type>
     182        </dependency>
     183        <dependency>
     184            <groupId>org.jgrapht</groupId>
     185            <artifactId>jgrapht-core</artifactId>
     186            <version>0.9.1</version>
    182187        </dependency>
    183188    </dependencies>

  • vlo/branches/vlo-ticket761/vlo-importer/src/main/bin/vlo_solr_importer.sh

    r6099 r6267  
    3636# take preference over a specification as a property.
    3737
    38 $JAVA -Xmx2024M \
     38$JAVA -Xmx3G \
    3939    -cp "${DIR}:${DIR}/vlo-importer-${project.version}-importer.jar" \
    4040    -DIMPORTER_LOG_DIR=${LOGDIR} \

  • vlo/branches/vlo-ticket761/vlo-importer/src/main/java/eu/clarin/cmdi/vlo/importer/CMDIParserVTDXML.java

    r6160 r6267  
    1212import java.io.FileInputStream;
    1313import java.io.IOException;
     14import java.net.URI;
    1415import java.util.ArrayList;
    1516import java.util.List;
    … …  
    2223
    2324public class CMDIParserVTDXML implements CMDIDataProcessor {
     25
    2426    private final Map<String, PostProcessor> postProcessors;
    2527    private final Boolean useLocalXSDCache;
    2628    private static final Pattern PROFILE_ID_PATTERN = Pattern.compile(".*(clarin.eu:cr1:p_[0-9]+).*");
    2729    private final static Logger LOG = LoggerFactory.getLogger(CMDIParserVTDXML.class);
    28    
     30
    2931    private static final String DEFAULT_LANGUAGE = "und";
    30    
     32
    3133    public CMDIParserVTDXML(Map<String, PostProcessor> postProcessors, Boolean useLocalXSDCache) {
    3234        this.postProcessors = postProcessors;
    … …  
    3840        CMDIData cmdiData = new CMDIData();
    3941        VTDGen vg = new VTDGen();
    40         FileInputStream fileInputStream = new FileInputStream(file); 
     42        FileInputStream fileInputStream = new FileInputStream(file);
    4143        vg.setDoc(IOUtils.toByteArray(fileInputStream));
    4244        vg.parse(true);
    4345        fileInputStream.close();
    44        
     46
    4547        VTDNav nav = vg.getNav();
    4648        FacetMapping facetMapping = getFacetMapping(nav.cloneNav());
    4749
    48         if(facetMapping.getFacets().isEmpty()){
     50        if (facetMapping.getFacets().isEmpty()) {
    4951            LOG.error("Problems mapping facets for file: {}", file.getAbsolutePath());
    5052        }
    … …  
    5860    /**
    5961     * Setting namespace for Autopilot ap
    60      * @param ap
     62     *
     63     * @param ap
    6164     */
    6265    private void setNameSpace(AutoPilot ap) {
    … …  
    6669    /**
    6770     * Extracts valid XML patterns for all facet definitions
     71     *
    6872     * @param nav VTD Navigator
    6973     * @return the facet mapping used to map meta data to facets
    70      * @throws VTDException 
     74     * @throws VTDException
    7175     */
    7276    private FacetMapping getFacetMapping(VTDNav nav) throws VTDException {
    … …  
    7579            throw new RuntimeException("Cannot get xsd schema so cannot get a proper mapping. Parse failed!");
    7680        }
     81//        final VloConfig config = MetadataImporter.config;
     82//        final URI facetConceptsFile
     83//                = FacetConceptsMarshaller.resolveFacetsFile(config.getConfigLocation(), config.getFacetConceptsFile());
     84//        final String facetConceptsFilePath = new File(facetConceptsFile).getAbsolutePath();
     85//        return FacetMappingFactory.getFacetMapping(facetConceptsFilePath, profileId, useLocalXSDCache);
     86
    7787        String facetConceptsFile = MetadataImporter.config.getFacetConceptsFile();
    78         if (facetConceptsFile.length() == 0){
    79             // use the packaged facet mapping file
    80             facetConceptsFile = "/facetConcepts.xml";
    81         }
     88
     89        //resolve against config location? (empty = default location)
     90        if (facetConceptsFile != null && !facetConceptsFile.isEmpty()) {
     91            URI configLocation = MetadataImporter.config.getConfigLocation();
     92            if (configLocation != null && !configLocation.getScheme().equals("jar")) {
     93                URI facetConceptsLocation = configLocation.resolve(facetConceptsFile);
     94                facetConceptsFile = new File(facetConceptsLocation).getAbsolutePath();
     95            }
     96        }
     97
    8298        return FacetMappingFactory.getFacetMapping(facetConceptsFile, profileId, useLocalXSDCache);
    8399    }
    84100
    85101    /**
    86      * Try two approaches to extract the XSD schema information from the CMDI file
    87      * @param nav VTD Navigator
    88      * @return ID of CMDI schema, or null if neither the CMDI header nor the XMLSchema-instance's attributes contained the information
    89      * @throws VTDException
     102     * Try two approaches to extract the XSD schema information from the CMDI
     103     * file
     104     *
     105     * @param nav VTD Navigator
     106     * @return ID of CMDI schema, or null if neither the CMDI header nor the
     107     * XMLSchema-instance's attributes contained the information
     108     * @throws VTDException
    90109     */
    91110    String extractXsd(VTDNav nav) throws VTDException {
    … …  
    98117
    99118    /**
    100      * Extract XSD schema information from CMDI header (using element //Header/MdProfile)
    101      * @param nav VTD Navigator
    102      * @return ID of CMDI schema, or null if content of //Header/MdProfile element could not be read
     119     * Extract XSD schema information from CMDI header (using element
     120     * //Header/MdProfile)
     121     *
     122     * @param nav VTD Navigator
     123     * @return ID of CMDI schema, or null if content of //Header/MdProfile
     124     * element could not be read
    103125     * @throws XPathParseException
    104126     * @throws XPathEvalException
    105      * @throws NavException 
     127     * @throws NavException
    106128     */
    107129    private String getProfileIdFromHeader(VTDNav nav) throws XPathParseException, XPathEvalException, NavException {
    … …  
    119141
    120142    /**
    121      * Extract XSD schema information from schemaLocation or noNamespaceSchemaLocation attributes
     143     * Extract XSD schema information from schemaLocation or
     144     * noNamespaceSchemaLocation attributes
     145     *
    122146     * @param nav VTD Navigator
    123147     * @return ID of CMDI schema, or null if attributes don't exist
    124      * @throws NavException 
     148     * @throws NavException
    125149     */
    126150    private String getProfileIdFromSchemaLocation(VTDNav nav) throws NavException {
    … …  
    137161            }
    138162        }
    139        
     163
    140164        // extract profile ID
    141         if(result != null) {
    142         Matcher m = PROFILE_ID_PATTERN.matcher(result);
    143         if(m.find())
    144             return m.group(1);
     165        if (result != null) {
     166            Matcher m = PROFILE_ID_PATTERN.matcher(result);
     167            if (m.find()) {
     168                return m.group(1);
     169            }
    145170        }
    146171        return null;
    147172    }
    148    
     173
    149174    /**
    150175     * Extract ResourceProxies from ResourceProxyList
     176     *
    151177     * @param cmdiData representation of the CMDI document
    152178     * @param nav VTD Navigator
    153      * @throws VTDException 
     179     * @throws VTDException
    154180     */
    155181    private void processResources(CMDIData cmdiData, VTDNav nav) throws VTDException {
    156        
     182        AutoPilot mdSelfLink = new AutoPilot(nav);
     183        setNameSpace(mdSelfLink);
     184        mdSelfLink.selectXPath("/c:CMD/c:Header/c:MdSelfLink");
     185        String mdSelfLinkString = mdSelfLink.evalXPathToString();
     186        ResourceStructureGraph.addResource(mdSelfLinkString);
     187
    157188        AutoPilot resourceProxy = new AutoPilot(nav);
    158189        setNameSpace(resourceProxy);
    159190        resourceProxy.selectXPath("/c:CMD/c:Resources/c:ResourceProxyList/c:ResourceProxy");
    160        
     191
    161192        AutoPilot resourceRef = new AutoPilot(nav);
    162193        setNameSpace(resourceRef);
    163194        resourceRef.selectXPath("c:ResourceRef");
    164        
     195
    165196        AutoPilot resourceType = new AutoPilot(nav);
    166197        setNameSpace(resourceType);
    167198        resourceType.selectXPath("c:ResourceType");
    168        
     199
    169200        AutoPilot resourceMimeType = new AutoPilot(nav);
    170201        setNameSpace(resourceMimeType);
    171202        resourceMimeType.selectXPath("c:ResourceType/@mimetype");
    172        
     203
    173204        while (resourceProxy.evalXPath() != -1) {
    174205            String ref = resourceRef.evalXPathToString();
    175206            String type = resourceType.evalXPathToString();
    176207            String mimeType = resourceMimeType.evalXPathToString();
    177            
     208
    178209            if (!ref.equals("") && !type.equals("")) {
    179210                // note that the mime type could be empty
    180211                cmdiData.addResource(ref, type, mimeType);
    181212            }
     213           
     214            // resource hierarchy information?
     215            if(type.toLowerCase().equals("metadata")) {
     216                ResourceStructureGraph.addEdge(ref, mdSelfLinkString);
     217            }
    182218        }
    183219    }
    … …  
    185221    /**
    186222     * Extracts facet values according to the facetMapping
     223     *
    187224     * @param cmdiData representation of the CMDI document
    188225     * @param nav VTD Navigator
    189226     * @param facetMapping the facet mapping used to map meta data to facets
    190      * @throws VTDException 
     227     * @throws VTDException
    191228     */
    192229    private void processFacets(CMDIData cmdiData, VTDNav nav, FacetMapping facetMapping) throws VTDException {
    … …  
    201238                }
    202239            }
    203            
     240
    204241            // using fallback patterns if extraction failed
    205242            if (matchedPattern == false) {
    … …  
    215252
    216253    /**
    217      * Extracts content from CMDI file for a specific facet based on a single XPath expression
     254     * Extracts content from CMDI file for a specific facet based on a single
     255     * XPath expression
     256     *
    218257     * @param cmdiData representation of the CMDI document
    219258     * @param nav VTD Navigator
    220259     * @param config facet configuration
    221260     * @param pattern XPath expression
    222      * @param allowMultipleValues information if multiple values are allowed in this facet
     261     * @param allowMultipleValues information if multiple values are allowed in
     262     * this facet
    223263     * @return pattern matched a node in the CMDI file?
    224      * @throws VTDException 
     264     * @throws VTDException
    225265     */
    226266    private boolean matchPattern(CMDIData cmdiData, VTDNav nav, FacetConfiguration config, String pattern, Boolean allowMultipleValues) throws VTDException {
    … …  
    237277            }
    238278            String value = nav.toString(index);
    239            
     279
    240280            // extract language code in xml:lang if available
    241281            Integer langAttrIndex = nav.getAttrVal("xml:lang");
    242282            String languageCode = DEFAULT_LANGUAGE;
    243             if(langAttrIndex != -1)
     283            if (langAttrIndex != -1) {
    244284                languageCode = nav.toString(langAttrIndex).trim();
     285            }
    245286            // replace 2-letter with 3-letter codes
    246             if(MetadataImporter.languageCodeUtils.getSilToIso639Map().containsKey(languageCode))
     287            if (MetadataImporter.languageCodeUtils.getSilToIso639Map().containsKey(languageCode)) {
    247288                languageCode = MetadataImporter.languageCodeUtils.getSilToIso639Map().get(languageCode);
    248            
     289            }
     290
    249291            // ignore non-English language names for facet LANGUAGE_CODE
    250             if(config.getName().equals(FacetConstants.FIELD_LANGUAGE_CODE) && !languageCode.equals("en") && !languageCode.equals("eng") && !languageCode.equals("und")) {
     292            if (config.getName().equals(FacetConstants.FIELD_LANGUAGE_CODE) && !languageCode.equals("en") && !languageCode.equals("eng") && !languageCode.equals("und")) {
    251293                index = ap.evalXPath();
    252                 continue; 
    253             }
    254            
     294                continue;
     295            }
     296
    255297            List<String> valueList = postProcess(config.getName(), value);
    256             for(int i=0; i<valueList.size(); i++) {
    257                 if(!allowMultipleValues && i>0)
     298            for (int i = 0; i < valueList.size(); i++) {
     299                if (!allowMultipleValues && i > 0) {
    258300                    break;
     301                }
    259302                String fieldValue = valueList.get(i).trim();
    260                 if(config.getName().equals(FacetConstants.FIELD_DESCRIPTION))
    261                     fieldValue = "{lang='"+languageCode+"'}"+fieldValue;
     303                if (config.getName().equals(FacetConstants.FIELD_DESCRIPTION)) {
     304                    fieldValue = "{lang='" + languageCode + "'}" + fieldValue;
     305                }
    262306                cmdiData.addDocField(config.getName(), fieldValue, config.isCaseInsensitive());
    263307            }
    264308            index = ap.evalXPath();
    265            
    266             if(!allowMultipleValues)
    267                 break;
     309
     310            if (!allowMultipleValues) {
     311                break;
     312            }
    268313        }
    269314        return matchedPattern;
    … …  
    272317    /**
    273318     * Applies registered PostProcessor to extracted values
     319     *
    274320     * @param facetName name of the facet for which value was extracted
    275321     * @param extractedValue extracted value from CMDI file
    276      * @return value after applying matching PostProcessor or the original value if no PostProcessor was registered for the facet
     322     * @return value after applying matching PostProcessor or the original value
     323     * if no PostProcessor was registered for the facet
    277324     */
    278325    private List<String> postProcess(String facetName, String extractedValue) {

  • vlo/branches/vlo-ticket761/vlo-importer/src/main/java/eu/clarin/cmdi/vlo/importer/FacetMappingFactory.java

    r5985 r6267  
    3636    }
    3737   
     38    /**
     39     *
     40     * @param facetConceptsFile path to facet concepts file, leave null or empty to use default
     41     * @param xsd
     42     * @param useLocalXSDCache
     43     * @return
     44     */
    3845    public static FacetMapping getFacetMapping(String facetConceptsFile, String xsd, Boolean useLocalXSDCache) {
    3946        return INSTANCE.getOrCreateMapping(facetConceptsFile, xsd, useLocalXSDCache);
    … …  
    166173                config.setName(facetConcept.getName());
    167174
    168                 config.setPatterns(xpaths);
     175                LinkedHashSet<String> linkedHashSet = new LinkedHashSet<String>(xpaths);
     176                if(xpaths.size() != linkedHashSet.size()) {
     177                    LOG.error("Duplicate XPaths in : "+xpaths);
     178                }
     179                config.setPatterns(new ArrayList<String>(linkedHashSet));
    169180                config.setFallbackPatterns(facetConcept.getPatterns());
    170181

  • vlo/branches/vlo-ticket761/vlo-importer/src/main/java/eu/clarin/cmdi/vlo/importer/IdPostProcessor.java

    r5345 r6267  
    11package eu.clarin.cmdi.vlo.importer;
    22
     3import eu.clarin.cmdi.vlo.StringUtils;
    34import java.util.ArrayList;
    4 import java.util.Arrays;
    5 import java.util.HashSet;
    65import java.util.List;
    7 import java.util.Set;
    86
    97public class IdPostProcessor implements PostProcessor {
    108
    11     private static final Set<Character> reservedCharacters = new HashSet<Character>(Arrays.asList('!', '*', '\'', '(',
    12             ')', ';', ':', '@', '&', '=', '+', '$', ',', '/', '?', '#', '[', ']'));
    13 
    149    /**
    15      * Return normalized String where all reserved characters in URL encoding
    16      * are replaced by their ASCII code (in underscores)
     10     * Return normalized String
    1711     *
    1812     * @param value String that will be normalized
    19      * @return normalized version of value where all reserved
    20      * characters in URL encoding are replaced by their ASCII code
     13     * @return normalized version of value
    2114     */
    2215    @Override
    2316    public List<String> process(String value) {
    24         StringBuilder normalizedString = new StringBuilder();
    25         for (int i = 0; i < value.length(); i++) {
    26             Character character = value.charAt(i);
    27             if (reservedCharacters.contains(character)) {
    28                 normalizedString.append("_").append((int) character).append("_");
    29             } else {
    30                 normalizedString.append(character);
    31             }
    32         }
    33 
    3417        List<String> resultList = new ArrayList<String>();
    35         resultList.add(normalizedString.toString());
     18        resultList.add(StringUtils.normalizeIdString(value));
    3619        return resultList;
    3720    }

  • vlo/branches/vlo-ticket761/vlo-importer/src/main/java/eu/clarin/cmdi/vlo/importer/MetadataImporter.java

    r6188 r6267  
    1414import java.text.SimpleDateFormat;
    1515import java.util.ArrayList;
    16 import java.util.Collection;
     16import java.util.Arrays;
    1717import java.util.Date;
    1818import java.util.HashMap;
    1919import java.util.HashSet;
     20import java.util.Iterator;
    2021import java.util.List;
    2122import java.util.Map;
    … …  
    137138                }
    138139                CMDIDataProcessor processor = new CMDIParserVTDXML(POST_PROCESSORS, false);
    139                 List<File> files = getFilesFromDataRoot(dataRoot.getRootFile());
    140                 for (File file : files) {
    141                     if (config.getMaxFileSize() > 0
    142                             && file.length() > config.getMaxFileSize()) {
    143                         LOG.info("Skipping " + file.getAbsolutePath() + " because it is too large.");
    144                     } else {
    145                         LOG.debug("PROCESSING FILE: {}", file.getAbsolutePath());
    146                         processCmdi(file, dataRoot, processor);
     140                List<List<File>> centreFilesList = getFilesFromDataRoot(dataRoot.getRootFile());
     141                // import files from every endpoint
     142                for(List<File> centreFiles : centreFilesList) {
     143                    LOG.info("Processing directory: {}", centreFiles.get(0).getParent());
     144                    for (File file : centreFiles) {
     145                        if (config.getMaxFileSize() > 0
     146                                && file.length() > config.getMaxFileSize()) {
     147                            LOG.info("Skipping " + file.getAbsolutePath() + " because it is too large.");
     148                        } else {
     149                            LOG.debug("PROCESSING FILE: {}", file.getAbsolutePath());
     150                            processCmdi(file, dataRoot, processor);
     151                        }
    147152                    }
    148                 }
    149                 if (!docs.isEmpty()) {
    150                     sendDocs();
     153                    if (!docs.isEmpty()) {
     154                        sendDocs();
     155                    }
     156                    updateDocumentHierarchy();
    151157                }
    152158                LOG.info("End of processing: " + dataRoot.getOriginName());
    … …  
    203209
    204210    /**
    205      * Get the rootFile or all files with VALID_CMDI_EXTENSIONS if rootFile is a
    206      * directory
     211     * Get all files with VALID_CMDI_EXTENSIONS if rootFile is a
     212     * directory that contains center directories or rootFile if it is a file
    207213     *
    208214     * @param rootFile
    209      * @return List with the rootFile or all contained files if rootFile is a
    210      * directory
    211      */
    212     protected List<File> getFilesFromDataRoot(File rootFile) {
    213         List<File> result = new ArrayList<File>();
    214         if (rootFile.isFile()) {
    215             result.add(rootFile);
     215     * @return List with centre Lists of all contained CMDI files if rootFile is a
     216     * directory or rootFile if it is a File
     217     */
     218    protected List<List<File>> getFilesFromDataRoot(File rootFile) {
     219        List<List<File>> result = new ArrayList<List<File>>();
     220        if(rootFile.isFile()) {
     221            List<File> singleFileList = new ArrayList<File>();
     222            singleFileList.add(rootFile);
     223            result.add(singleFileList);
    216224        } else {
    217             Collection<File> listFiles = FileUtils.listFiles(rootFile, VALID_CMDI_EXTENSIONS, true);
    218             result.addAll(listFiles);
     225            File[] centerDirs = rootFile.listFiles();
     226            for(File centerDir : centerDirs) {
     227                List<File> centerFileList = new ArrayList<File>();
     228                if(centerDir.isDirectory()) {
     229                    centerFileList.addAll(FileUtils.listFiles(centerDir, VALID_CMDI_EXTENSIONS, true));
     230                }
     231               
     232                if(!centerFileList.isEmpty())
     233                    result.add(centerFileList);
     234            }
    219235        }
    220236        return result;
    … …  
    436452    }
    437453   
     454    /**
     455     * Updates documents in Solr with their hierarchy weight and lists of related resources (hasPart & isPartOf)
     456     * @throws SolrServerException
     457     * @throws MalformedURLException
     458     */
     459    private void updateDocumentHierarchy() throws SolrServerException, MalformedURLException, IOException {
     460        LOG.info(ResourceStructureGraph.printStatistics(0));
     461        List<SolrInputDocument> updateDocs = new ArrayList<SolrInputDocument>();
     462        Iterator<CmdiVertex> vertexIter = ResourceStructureGraph.getFoundVertices().iterator();
     463        while(vertexIter.hasNext()) {
     464            CmdiVertex vertex = vertexIter.next();
     465            List<String> incomingVertexNames = ResourceStructureGraph.getIncomingVertexNames(vertex);
     466            List<String> outgoingVertexNames = ResourceStructureGraph.getOutgoingVertexNames(vertex);
     467           
     468            // update vertex if changes are necessary (necessary if non-default weight or edges to other resources)
     469            if(vertex.getHierarchyWeight() != 0 || !incomingVertexNames.isEmpty() || !outgoingVertexNames.isEmpty()) {
     470                SolrInputDocument doc = new SolrInputDocument();
     471                doc.setField(FacetConstants.FIELD_ID, Arrays.asList(vertex.getId()));
     472               
     473                if(vertex.getHierarchyWeight() != 0) {
     474                    Map<String, Integer> partialUpdate = new HashMap<String, Integer>();
     475                    partialUpdate.put("set", Math.abs(vertex.getHierarchyWeight()));
     476                    doc.addField(FacetConstants.FIELD_HIERARCHY_WEIGHT, partialUpdate);
     477                }
     478               
     479                if(!incomingVertexNames.isEmpty()) {
     480                    Map<String, List<String>> partialUpdate = new HashMap<String, List<String>>();
     481                    partialUpdate.put("set", incomingVertexNames);
     482                    doc.setField(FacetConstants.FIELD_HAS_PART, partialUpdate);
     483                }
     484               
     485                if(!outgoingVertexNames.isEmpty()) {
     486                    Map<String, List<String>> partialUpdate = new HashMap<String, List<String>>();
     487                    partialUpdate.put("set", outgoingVertexNames);
     488                    doc.setField(FacetConstants.FIELD_IS_PART_OF, partialUpdate);
     489                }
     490                updateDocs.add(doc);
     491            }
     492           
     493            if (updateDocs.size() == config.getMaxDocsInList()) {
     494                solrServer.add(updateDocs);
     495                if (serverError != null) {
     496                    throw new SolrServerException(serverError);
     497                }
     498                updateDocs = new ArrayList<SolrInputDocument>();
     499            }
     500        }
     501        if(!updateDocs.isEmpty()) {
     502            solrServer.add(updateDocs);
     503            if (serverError != null) {
     504                throw new SolrServerException(serverError);
     505            }
     506        }
     507        solrServer.commit();
     508
     509        ResourceStructureGraph.clearResourceGraph();
     510    }
     511   
    438512    public static VloConfig config;
    439513

  • vlo/branches/vlo-ticket761/vlo-importer/src/main/java/eu/clarin/cmdi/vlo/importer/TemporalCoveragePostProcessor.java

    r6188 r6267  
    99
    1010    // Open Date Range Format
    11     private static final Pattern ODRF_PATTERN = Pattern.compile("(([0-9]{4}(-[0-9]{2})?(-[0-9]{2})?)?/([0-9]{4}(-[0-9]{2})?(-[0-9]{2})?)?)");
    12     // simple year
    13     private static final Pattern YEAR_PATTERN = Pattern.compile("([0-9]{4}(-[0-9]{2})?(-[0-9]{2})?)");
     11    private static final Pattern ODRF_PATTERN = Pattern.compile("^(([0-9]{4}(-[0-9]{2})?(-[0-9]{2})?)?/([0-9]{4}(-[0-9]{2})?(-[0-9]{2})?)?)");
     12    // Simplified W3C DateTime
     13    private static final Pattern DATETIME_PATTERN = Pattern.compile("^([0-9]{4}(-[0-9]{2})?(-[0-9]{2})?)");
    1414
    1515    /**
    16      * Tries to identify relevant year substrings in input
     16     * Tries to identify relevant temporal substrings in input
    1717     *
    18      * @param value extracted year or date range String
     18     * @param value extracted date or date range String
    1919     * @return List of accepted values
    2020     */
    2121    @Override
    2222    public List<String> process(final String value) {
    23         Matcher odrfMatcher = ODRF_PATTERN.matcher(value);
    24         Matcher yearMatcher = YEAR_PATTERN.matcher(value);
     23        String coverageString = value.trim();
     24       
     25        Matcher odrfMatcher = ODRF_PATTERN.matcher(coverageString);
     26        Matcher yearMatcher = DATETIME_PATTERN.matcher(coverageString);
    2527        List<String> resultList = new ArrayList<String>();
    2628

  • vlo/branches/vlo-ticket761/vlo-importer/src/main/java/eu/clarin/cmdi/vlo/importer/VLOMarshaller.java

    r4029 r6267  
    11package eu.clarin.cmdi.vlo.importer;
    22
     3import eu.clarin.cmdi.vlo.config.VloConfig;
     4import java.io.FileInputStream;
     5import java.io.FileNotFoundException;
    36import java.io.InputStream;
    47import java.io.StringWriter;
    … …  
    811import javax.xml.bind.Marshaller;
    912import javax.xml.bind.Unmarshaller;
     13import org.slf4j.Logger;
     14import org.slf4j.LoggerFactory;
    1015
    1116public class VLOMarshaller {
    1217
     18    private final static Logger logger = LoggerFactory.getLogger(VLOMarshaller.class);
     19
    1320    /**
    1421     * Get facet concepts mapping from a facet concept mapping file
    15      * 
     22     *
    1623     * @param facetConcepts name of the facet concepts file
    1724     * @return the facet concept mapping
    … …  
    2027            String facetConcepts) {
    2128
    22         InputStream inputStream =
    23                 VLOMarshaller.class.getResourceAsStream(facetConcepts);
    24        
    25         return unmarshal(inputStream);
     29        if (facetConcepts == null || "".equals(facetConcepts)) {
     30            return unmarshal(VLOMarshaller.class.getResourceAsStream(VloConfig.DEFAULT_FACET_CONCEPTS_RESOURCE_FILE));
     31        } else {
     32            try {
     33                return unmarshal(new FileInputStream(facetConcepts));
     34            } catch (FileNotFoundException ex) {
     35                logger.error("Could not find facets file: {}", facetConcepts);
     36                return null;
     37            }
     38        }
    2639    }
    2740
    2841    /**
    2942     * Get object from input stream
    30      * 
     43     *
    3144     * @param inputStream
    32      * @return 
     45     * @return
    3346     */
    3447    static FacetConceptMapping unmarshal(InputStream inputStream) {
    … …  
    4659    /**
    4760     * Put facet mapping object in output file
    48      * 
     61     *
    4962     * @param outputFile
    50      * @return 
     63     * @return
    5164     */
    5265    static String marshal(FacetConceptMapping outputFile) {

  • vlo/branches/vlo-ticket761/vlo-importer/src/test/java/eu/clarin/cmdi/vlo/importer/CMDIDataProcessorTest.java

    r6188 r6267  
    33import eu.clarin.cmdi.vlo.FacetConstants;
    44import java.io.File;
     5import java.net.URL;
    56import java.util.ArrayList;
    67import java.util.Collection;
    … …  
    1213import static org.junit.Assert.assertNull;
    1314import static org.junit.Assert.assertTrue;
     15import org.junit.Before;
    1416import org.junit.Test;
    1517
    … …  
    1921        return new CMDIParserVTDXML(MetadataImporter.POST_PROCESSORS, true);
    2022    }
    21    
     23
     24    @Before
     25    @Override
     26    public void setup() throws Exception {
     27        super.setup();
     28        // make sure the mapping file for testing is used
     29        config.setFacetConceptsFile(getTestFacetConceptFilePath());
     30    }
     31
    2232    @Test
    2333    public void testCreateCMDIDataFromCorpus() throws Exception {
    24        
    25         // make sure the mapping file for testing is used
    26         config.setFacetConceptsFile("/facetConceptsTest.xml");
    27 
    2834        String content = "";
    2935        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    8692    @Test
    8793    public void testCreateCMDIDataFromSession() throws Exception {
    88        
    89         // make sure the mapping file for testing is used
    90         config.setFacetConceptsFile("/facetConceptsTest.xml");
    91 
    9294        String content = "";
    9395        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    412414    @Test
    413415    public void testCreateCMDISessionSmall() throws Exception {
    414        
    415         // make sure the mapping file for testing is used
    416         config.setFacetConceptsFile("/facetConceptsTest.xml");
    417 
    418416        String content = "";
    419417        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    441439    @Test
    442440    public void testEmptyFieldsShouldBeNull() throws Exception {
    443        
    444         // make sure the mapping file for testing is used
    445         config.setFacetConceptsFile("/facetConceptsTest.xml");
    446 
    447441        String content = "";
    448442        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    534528    @Test
    535529    public void testOlac() throws Exception {
    536 
    537         // make sure the mapping file for testing is used
    538         config.setFacetConceptsFile("/facetConceptsTest.xml");
    539 
    540530        String content = "";
    541531        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    621611    @Test
    622612    public void testOlacMultiFacets() throws Exception {
    623        
    624         // make sure the mapping file for testing is used
    625         config.setFacetConceptsFile("/facetConceptsTest.xml");
    626 
    627613        String content = "";
    628614        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    717703    @Test
    718704    public void testIgnoreWhiteSpaceFacets() throws Exception {
    719        
    720         // make sure the mapping file for testing is used
    721         config.setFacetConceptsFile("/facetConceptsTest.xml");
    722 
    723705        String content = "";
    724706        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    743725    @Test
    744726    public void testCountryCodesPostProcessing() throws Exception {
    745        
    746         // make sure the mapping file for testing is used
    747         config.setFacetConceptsFile("/facetConceptsTest.xml");
    748 
    749727        String content = "";
    750728        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    769747    @Test
    770748    public void testLanguageCodesPostProcessing() throws Exception {
    771        
    772         // make sure the mapping file for testing is used
    773         config.setFacetConceptsFile("/facetConceptsTest.xml");
    774 
    775749        String content = "";
    776750        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    800774    @Test
    801775    public void testOlacCollection() throws Exception {
    802        
    803         // make sure the mapping file for testing is used
    804         config.setFacetConceptsFile("/facetConceptsTest.xml");
    805 
    806776        String content = "";
    807777        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    … …  
    853823    @Test
    854824    public void testLrtCollection() throws Exception {
    855        
    856         // make sure the mapping file for testing is used
    857         config.setFacetConceptsFile("/facetConceptsTest.xml");
    858 
    859825        String content = "";
    860826        content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";

  • vlo/branches/vlo-ticket761/vlo-importer/src/test/java/eu/clarin/cmdi/vlo/importer/FacetMappingFactoryTest.java

    r6188 r6267  
    1313
    1414public class FacetMappingFactoryTest {
    15    
    16     private final static String FACETCONCEPTS_FILENAME = "/facetConceptsTest.xml";
    17    
     15
     16    private final static String FACETCONCEPTS_FILENAME = ImporterTestcase.getTestFacetConceptFilePath();
     17
    1818    private final static String IMDI_PROFILE_ID = "clarin.eu:cr1:p_1271859438204";
    1919    private final static String OLAC_PROFILE_ID = "clarin.eu:cr1:p_1288172614026";
    … …  
    2828        FacetMapping facetMapping = FacetMappingFactory
    2929                .getFacetMapping(FACETCONCEPTS_FILENAME, IMDI_PROFILE_ID, true);
    30        
     30
    3131        List<FacetConfiguration> facets = facetMapping.getFacets();
    3232        assertEquals(20, facets.size());
    33        
     33
    3434        int index = 0;
    3535        FacetConfiguration mapping = facets.get(index++);
    36        
     36
    3737        assertEquals(FacetConstants.FIELD_ID, mapping.getName());
    3838        assertEquals(1, mapping.getPatterns().size());
    39         assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()", 
    40                 mapping.getPatterns().get(0));
    41         mapping = facets.get(index++);
    42        
     39        assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()",
     40                mapping.getPatterns().get(0));
     41        mapping = facets.get(index++);
     42
    4343        assertEquals(FacetConstants.FIELD_SELF_LINK, mapping.getName());
    4444        assertEquals(0, mapping.getPatterns().size());
    4545        assertEquals(1, mapping.getFallbackPatterns().size());
    46         assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()", 
    47                 mapping.getFallbackPatterns().get(0));
    48         mapping = facets.get(index++);
    49        
     46        assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()",
     47                mapping.getFallbackPatterns().get(0));
     48        mapping = facets.get(index++);
     49
    5050        assertEquals(FacetConstants.FIELD_COLLECTION, mapping.getName());
    5151        assertEquals(1, mapping.getFallbackPatterns().size());
    5252        mapping = facets.get(index++);
    53        
     53
    5454        assertEquals(FacetConstants.FIELD_PROJECT_NAME, mapping.getName());
    5555        assertEquals(2, mapping.getPatterns().size());
    56         assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Project/c:Name/text()", 
    57                 mapping.getPatterns().get(0));
    58         assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Project/c:Title/text()", 
    59                 mapping.getPatterns().get(1));
    60         mapping = facets.get(index++);
    61        
     56        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Project/c:Name/text()",
     57                mapping.getPatterns().get(0));
     58        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Project/c:Title/text()",
     59                mapping.getPatterns().get(1));
     60        mapping = facets.get(index++);
     61
    6262        assertEquals(FacetConstants.FIELD_NAME, mapping.getName());
    6363        assertEquals(2, mapping.getPatterns().size());
    64         assertEquals("/c:CMD/c:Components/c:Session/c:Name/text()", 
    65                 mapping.getPatterns().get(0));
    66         assertEquals("/c:CMD/c:Components/c:Session/c:Title/text()", 
    67                 mapping.getPatterns().get(1));
    68         mapping = facets.get(index++);
    69        
     64        assertEquals("/c:CMD/c:Components/c:Session/c:Name/text()",
     65                mapping.getPatterns().get(0));
     66        assertEquals("/c:CMD/c:Components/c:Session/c:Title/text()",
     67                mapping.getPatterns().get(1));
     68        mapping = facets.get(index++);
     69
    7070        assertEquals(FacetConstants.FIELD_TEMPORAL_COVERAGE, mapping.getName());
    7171        assertEquals(2, mapping.getPatterns().size());
    72         assertEquals("/c:CMD/c:Components/c:Session/c:Date/text()", 
    73                 mapping.getPatterns().get(0));
    74         mapping = facets.get(index++);
    75        
     72        assertEquals("/c:CMD/c:Components/c:Session/c:Date/text()",
     73                mapping.getPatterns().get(0));
     74        mapping = facets.get(index++);
     75
    7676        assertEquals(FacetConstants.FIELD_CONTINENT, mapping.getName());
    7777        assertEquals(1, mapping.getPatterns().size());
    78         assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Location/c:Continent/text()", 
     78        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Location/c:Continent/text()",
    7979                mapping.getPatterns()
    8080                .get(0));
    8181        mapping = facets.get(index++);
    82        
     82
    8383        assertEquals(FacetConstants.FIELD_COUNTRY, mapping.getName());
    8484        assertEquals(1, mapping.getPatterns().size());
    85         assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Location/c:Country/text()", 
     85        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Location/c:Country/text()",
    8686                mapping.getPatterns()
    8787                .get(0));
    8888        mapping = facets.get(index++);
    89        
     89
    9090        assertEquals(FacetConstants.FIELD_LANGUAGE_CODE, mapping.getName());
    9191        assertEquals(2, mapping.getPatterns().size());
    … …  
    9595        // assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Actors/c:Actor/c:Actor_Languages/c:Actor_Language/c:Id/text()",
    9696        // mapping.getPatterns().get(1));
    97         assertEquals("/c:CMD/c:Components/c:Session/c:Resources/c:WrittenResource/c:LanguageId/text()", 
    98                 mapping.getPatterns().get(1));
    99         mapping = facets.get(index++);
    100        
     97        assertEquals("/c:CMD/c:Components/c:Session/c:Resources/c:WrittenResource/c:LanguageId/text()",
     98                mapping.getPatterns().get(1));
     99        mapping = facets.get(index++);
     100
    101101        assertEquals(FacetConstants.FIELD_ORGANISATION, mapping.getName());
    102102        assertEquals(6, mapping.getPatterns().size());
    103         assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Project/c:Contact/c:Organisation/text()", 
     103        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Project/c:Contact/c:Organisation/text()",
    104104                mapping.getPatterns().get(0));
    105105        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Actors/c:Actor/c:Contact/c:Organisation/text()",
    … …  
    114114                mapping.getPatterns().get(5));
    115115        mapping = facets.get(index++);
    116        
     116
    117117        assertEquals(FacetConstants.FIELD_GENRE, mapping.getName());
    118118        assertEquals(2, mapping.getPatterns().size());
    119         assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Content/c:Genre/text()", 
    120                 mapping.getPatterns().get(0));
    121         mapping = facets.get(index++);
    122        
     119        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Content/c:Genre/text()",
     120                mapping.getPatterns().get(0));
     121        mapping = facets.get(index++);
     122
    123123        assertEquals(FacetConstants.FIELD_MODALITY, mapping.getName());
    124124        assertEquals(1, mapping.getPatterns().size());
    125125        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Content/c:Modalities/text()", mapping.getPatterns().get(0));
    126126        mapping = facets.get(index++);
    127        
     127
    128128        assertEquals(FacetConstants.FIELD_SUBJECT, mapping.getName());
    129129        assertEquals(1, mapping.getPatterns().size());
    130         assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Content/c:Subject/text()", 
    131                 mapping.getPatterns().get(0));
    132         mapping = facets.get(index++);
    133        
     130        assertEquals("/c:CMD/c:Components/c:Session/c:MDGroup/c:Content/c:Subject/text()",
     131                mapping.getPatterns().get(0));
     132        mapping = facets.get(index++);
     133
    134134        assertEquals(FacetConstants.FIELD_DESCRIPTION, mapping.getName());
    135135        assertEquals(18, mapping.getPatterns().size());
    136         assertEquals("/c:CMD/c:Components/c:Session/c:descriptions/c:Description/text()", 
    137                 mapping.getPatterns().get(0));
    138         mapping = facets.get(index++);
    139        
     136        assertEquals("/c:CMD/c:Components/c:Session/c:descriptions/c:Description/text()",
     137                mapping.getPatterns().get(0));
     138        mapping = facets.get(index++);
     139
    140140        assertEquals(FacetConstants.FIELD_RESOURCE_CLASS, mapping.getName());
    141141        assertEquals(3, mapping.getFallbackPatterns().size());
    142142        mapping = facets.get(index++);
    143        
     143
    144144        assertEquals(FacetConstants.FIELD_FORMAT, mapping.getName());
    145145        assertEquals(1, mapping.getPatterns().size());
    146146        mapping = facets.get(index++);
    147        
    148         assertEquals("/c:CMD/c:Header/c:MdCollectionDisplayName/text()", 
    149                 mapping.getFallbackPatterns().get(0));
    150         assertEquals(1, mapping.getFallbackPatterns().size());
    151         mapping = facets.get(index++);
    152        
     147
     148        assertEquals("/c:CMD/c:Header/c:MdCollectionDisplayName/text()",
     149                mapping.getFallbackPatterns().get(0));
     150        assertEquals(1, mapping.getFallbackPatterns().size());
     151        mapping = facets.get(index++);
     152
    153153        assertEquals("/c:CMD/c:Header//text()", mapping.getFallbackPatterns().get(0));
    154154        assertEquals("/c:CMD/c:Components//text()", mapping.getFallbackPatterns().get(1));
    155155        assertEquals(2, mapping.getFallbackPatterns().size());
    156156        mapping = facets.get(index++);
    157        
     157
    158158        assertEquals("/c:CMD/c:Header/c:MdProfile/text()", mapping.getFallbackPatterns().get(0));
    159159        assertEquals(1, mapping.getFallbackPatterns().size());
    … …  
    162162        // test keywords facet mapping
    163163        assertEquals(FacetConstants.FIELD_KEYWORDS, mapping.getName());
    164         assertEquals("/c:CMD/c:Components/c:mods/c:classification/text()", 
     164        assertEquals("/c:CMD/c:Components/c:mods/c:classification/text()",
    165165                mapping.getFallbackPatterns().get(0));
    166166        assertEquals(3, mapping.getFallbackPatterns().size());
    … …  
    173173        FacetMapping facetMapping = FacetMappingFactory
    174174                .getFacetMapping(FACETCONCEPTS_FILENAME, OLAC_PROFILE_ID, true);
    175        
     175
    176176        List<FacetConfiguration> facets = facetMapping.getFacets();
    177177        assertEquals(18, facets.size());
    178        
     178
    179179        int index = 0;
    180180        FacetConfiguration mapping = facets.get(index++);
    181        
     181
    182182        assertEquals(FacetConstants.FIELD_ID, mapping.getName());
    183183        assertEquals(1, mapping.getPatterns().size());
    184         assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()", 
    185                 mapping.getPatterns().get(0));
    186         mapping = facets.get(index++);
    187        
     184        assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()",
     185                mapping.getPatterns().get(0));
     186        mapping = facets.get(index++);
     187
    188188        assertEquals(FacetConstants.FIELD_SELF_LINK, mapping.getName());
    189189        assertEquals(0, mapping.getPatterns().size());
    190190        assertEquals(1, mapping.getFallbackPatterns().size());
    191         assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()", 
    192                 mapping.getFallbackPatterns().get(0));
    193         mapping = facets.get(index++);
    194        
     191        assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()",
     192                mapping.getFallbackPatterns().get(0));
     193        mapping = facets.get(index++);
     194
    195195        assertEquals(FacetConstants.FIELD_COLLECTION, mapping.getName());
    196196        assertEquals(1, mapping.getFallbackPatterns().size());
    197         assertEquals("/c:CMD/c:Header/c:MdCollectionDisplayName/text()", 
    198                 mapping.getFallbackPatterns().get(0));
    199         mapping = facets.get(index++);
    200        
     197        assertEquals("/c:CMD/c:Header/c:MdCollectionDisplayName/text()",
     198                mapping.getFallbackPatterns().get(0));
     199        mapping = facets.get(index++);
     200
    201201        assertEquals(FacetConstants.FIELD_PROJECT_NAME, mapping.getName());
    202202        assertEquals(3, mapping.getFallbackPatterns().size());
    203         assertEquals("/c:CMD/c:Components/c:teiHeader/c:fileDesc/c:publicationStmt/c:publisher/c:orgName/c:orgName[@role=\"project\"]/text()", 
    204                 mapping.getFallbackPatterns().get(0));
    205         mapping = facets.get(index++);
    206        
     203        assertEquals("/c:CMD/c:Components/c:teiHeader/c:fileDesc/c:publicationStmt/c:publisher/c:orgName/c:orgName[@role=\"project\"]/text()",
     204                mapping.getFallbackPatterns().get(0));
     205        mapping = facets.get(index++);
     206
    207207        assertEquals(FacetConstants.FIELD_NAME, mapping.getName());
    208208        assertEquals(1, mapping.getPatterns().size());
    209         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:title/text()", 
    210                 mapping.getPatterns().get(0));
    211         mapping = facets.get(index++);
    212        
     209        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:title/text()",
     210                mapping.getPatterns().get(0));
     211        mapping = facets.get(index++);
     212
    213213        assertEquals(FacetConstants.FIELD_TEMPORAL_COVERAGE, mapping.getName());
    214214        assertEquals(3, mapping.getPatterns().size());
    215         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:created/text()", 
    216                 mapping.getPatterns().get(0));
    217         mapping = facets.get(index++);
    218        
     215        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:created/text()",
     216                mapping.getPatterns().get(0));
     217        mapping = facets.get(index++);
     218
    219219        assertEquals(FacetConstants.FIELD_COUNTRY, mapping.getName());
    220220        assertEquals(0, mapping.getPatterns().size());
    221221        assertEquals(2, mapping.getFallbackPatterns().size());
    222         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:spatial[@dcterms-type=\"ISO3166\"]/text()", 
     222        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:spatial[@dcterms-type=\"ISO3166\"]/text()",
    223223                mapping
    224224                .getFallbackPatterns().get(0));
    225         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:coverage[@dcterms-type=\"ISO3166\"]/text()", 
     225        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:coverage[@dcterms-type=\"ISO3166\"]/text()",
    226226                mapping
    227227                .getFallbackPatterns().get(1));
    228228        mapping = facets.get(index++);
    229        
     229
    230230        assertEquals(FacetConstants.FIELD_LANGUAGE_CODE, mapping.getName());
    231231        assertEquals(1, mapping.getPatterns().size());
    … …  
    234234        assertEquals("/c:CMD/c:Components//c:OLAC-DcmiTerms/c:language/@olac-language", mapping.getFallbackPatterns().get(0));
    235235        mapping = facets.get(index++);
    236        
     236
    237237        assertEquals(FacetConstants.FIELD_ORGANISATION, mapping.getName());
    238238        assertEquals(1, mapping.getPatterns().size());
    239         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:publisher/text()", 
     239        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:publisher/text()",
    240240                mapping.getPatterns().get(0));
    241241        mapping = facets.get(index++);
    … …  
    243243        assertEquals(FacetConstants.FIELD_GENRE, mapping.getName());
    244244        assertEquals(4, mapping.getFallbackPatterns().size());
    245         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:type/@olac-linguistic-type", 
    246                 mapping.getFallbackPatterns().get(0));
    247         assertEquals("/c:CMD/c:Components/c:mods/c:genre/text()", 
     245        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:type/@olac-linguistic-type",
     246                mapping.getFallbackPatterns().get(0));
     247        assertEquals("/c:CMD/c:Components/c:mods/c:genre/text()",
    248248                mapping.getFallbackPatterns().get(1));
    249249        mapping = facets.get(index++);
    … …  
    251251        assertEquals(FacetConstants.FIELD_SUBJECT, mapping.getName());
    252252        assertEquals(1, mapping.getPatterns().size());
    253         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:subject/text()", 
     253        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:subject/text()",
    254254                mapping.getPatterns().get(0));
    255255        // assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:subject[@dcterms-type=\"LCSH\"]/text()", mapping.getPatterns().get(1));
    256256        mapping = facets.get(index++);
    257        
     257
    258258        assertEquals(FacetConstants.FIELD_DESCRIPTION, mapping.getName());
    259259        assertEquals(1, mapping.getPatterns().size());
    260         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:description/text()", 
    261                 mapping.getPatterns().get(0));
    262         mapping = facets.get(index++);
    263        
     260        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:description/text()",
     261                mapping.getPatterns().get(0));
     262        mapping = facets.get(index++);
     263
    264264        assertEquals(FacetConstants.FIELD_RESOURCE_CLASS, mapping.getName());
    265         assertEquals(1, mapping.getPatterns().size());       
    266         mapping = facets.get(index++);
    267        
     265        assertEquals(1, mapping.getPatterns().size());
     266        mapping = facets.get(index++);
     267
    268268        assertEquals(FacetConstants.FIELD_FORMAT, mapping.getName());
    269         assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:format/text()", 
    270                 mapping.getPatterns().get(0));
    271         assertEquals(1, mapping.getPatterns().size());
    272         mapping = facets.get(index++);
    273        
     269        assertEquals("/c:CMD/c:Components/c:OLAC-DcmiTerms/c:format/text()",
     270                mapping.getPatterns().get(0));
     271        assertEquals(1, mapping.getPatterns().size());
     272        mapping = facets.get(index++);
     273
    274274        assertEquals(FacetConstants.FIELD_NATIONAL_PROJECT, mapping.getName());
    275275        assertEquals(1, mapping.getFallbackPatterns().size());
    276276        mapping = facets.get(index++);
    277        
    278277
    279278        assertEquals("/c:CMD/c:Header//text()", mapping.getFallbackPatterns().get(0));
    … …  
    281280        assertEquals(2, mapping.getFallbackPatterns().size());
    282281        mapping = facets.get(index++);
    283        
    284        
     282
    285283        assertEquals("/c:CMD/c:Header/c:MdProfile/text()", mapping.getFallbackPatterns().get(0));
    286284        assertEquals(1, mapping.getFallbackPatterns().size());
    … …  
    288286
    289287        assertEquals(FacetConstants.FIELD_KEYWORDS, mapping.getName());
    290         assertEquals("/c:CMD/c:Components/c:mods/c:classification/text()", 
     288        assertEquals("/c:CMD/c:Components/c:mods/c:classification/text()",
    291289                mapping.getFallbackPatterns().get(0));
    292290        assertEquals(3, mapping.getFallbackPatterns().size());
    … …  
    302300        List<FacetConfiguration> facets = facetMapping.getFacets();
    303301        assertEquals(17, facets.size());
    304        
     302
    305303        int index = 0;
    306304        FacetConfiguration mapping = facets.get(index++);
    307        
     305
    308306        assertEquals(FacetConstants.FIELD_ID, mapping.getName());
    309307        assertEquals(1, mapping.getPatterns().size());
    310        
    311         assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()", 
    312                 mapping.getPatterns().get(0));
    313         mapping = facets.get(index++);
    314        
     308
     309        assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()",
     310                mapping.getPatterns().get(0));
     311        mapping = facets.get(index++);
     312
    315313        assertEquals(FacetConstants.FIELD_SELF_LINK, mapping.getName());
    316314        assertEquals(0, mapping.getPatterns().size());
    317315        assertEquals(1, mapping.getFallbackPatterns().size());
    318         assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()", 
    319                 mapping.getFallbackPatterns().get(0));
    320         mapping = facets.get(index++);
    321        
     316        assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()",
     317                mapping.getFallbackPatterns().get(0));
     318        mapping = facets.get(index++);
     319
    322320        assertEquals(FacetConstants.FIELD_COLLECTION, mapping.getName());
    323321        assertEquals(1, mapping.getFallbackPatterns().size());
    324322        mapping = facets.get(index++);
    325        
     323
    326324        assertEquals(FacetConstants.FIELD_PROJECT_NAME, mapping.getName());
    327325        assertEquals(3, mapping.getFallbackPatterns().size());
    328326        assertEquals("/c:CMD/c:Components/c:media-session-profile/c:media-session/c:Corpus", mapping.getFallbackPatterns().get(1));
    329327        mapping = facets.get(index++);
    330        
     328
    331329        assertEquals(FacetConstants.FIELD_NAME, mapping.getName());
    332330        assertEquals(6, mapping.getFallbackPatterns().size());
    333         assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:ResourceName/text()", 
    334                 mapping.getFallbackPatterns().get(0));
    335         assertEquals("/c:CMD/c:Components/c:mods/c:titleInfo/title/text()", 
     331        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:ResourceName/text()",
     332                mapping.getFallbackPatterns().get(0));
     333        assertEquals("/c:CMD/c:Components/c:mods/c:titleInfo/title/text()",
    336334                mapping.getFallbackPatterns().get(1));
    337335        mapping = facets.get(index++);
    338        
     336
    339337        assertEquals(FacetConstants.FIELD_TEMPORAL_COVERAGE, mapping.getName());
    340338        assertEquals(3, mapping.getPatterns().size());
    … …  
    342340                "/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:FinalizationYearResourceCreation/text()",
    343341                mapping.getPatterns().get(0));
    344         assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtLexiconDetails/c:Date/text()", 
    345                 mapping.getPatterns().get(1));
    346         assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtServiceDetails/c:Date/text()", 
     342        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtLexiconDetails/c:Date/text()",
     343                mapping.getPatterns().get(1));
     344        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtServiceDetails/c:Date/text()",
    347345                mapping.getPatterns().get(2));
    348346        mapping = facets.get(index++);
    349        
     347
    350348        assertEquals(FacetConstants.FIELD_COUNTRY, mapping.getName());
    351349        assertEquals(2, mapping.getPatterns().size());
    352         assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:Countries/c:Country/text()", 
     350        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:Countries/c:Country/text()",
    353351                mapping.getPatterns().get(0));
    354352        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:Countries/c:Country/c:Code/text()",
    355353                mapping.getPatterns().get(1));
    356354        mapping = facets.get(index++);
    357        
     355
    358356        assertEquals(FacetConstants.FIELD_LANGUAGE_CODE, mapping.getName());
    359357
    … …  
    366364                mapping.getPatterns().get(0));
    367365        mapping = facets.get(index++);
    368        
     366
    369367        assertEquals(FacetConstants.FIELD_ORGANISATION, mapping.getName());
    370368        assertEquals(1, mapping.getPatterns().size());
    371         assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:Institute/text()", 
     369        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:Institute/text()",
    372370                mapping.getPatterns()
    373371                .get(0));
    374372        mapping = facets.get(index++);
    375        
     373
    376374        assertEquals(FacetConstants.FIELD_GENRE, mapping.getName());
    377375        assertEquals(4, mapping.getFallbackPatterns().size());
    378376        mapping = facets.get(index++);
    379        
     377
    380378        assertEquals(FacetConstants.FIELD_SUBJECT, mapping.getName());
    381379        assertEquals(8, mapping.getFallbackPatterns().size());
    382380        mapping = facets.get(index++);
    383        
     381
    384382        assertEquals(FacetConstants.FIELD_DESCRIPTION, mapping.getName());
    385383        assertEquals(2, mapping.getPatterns().size());
    386384        assertEquals(5, mapping.getFallbackPatterns().size());
    387         assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:Description/text()", 
     385        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:Description/text()",
    388386                mapping.getPatterns().get(0));
    389387        assertEquals("/c:CMD/c:Components/c:mods/c:abstract/text()", mapping.getFallbackPatterns().get(0));
    390388        mapping = facets.get(index++);
    391        
     389
    392390        assertEquals(FacetConstants.FIELD_RESOURCE_CLASS, mapping.getName());
    393391        assertEquals(1, mapping.getPatterns().size());
    394392        assertEquals(3, mapping.getFallbackPatterns().size());
    395         assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:ResourceType/text()", 
    396                 mapping.getFallbackPatterns().get(0));
    397         mapping = facets.get(index++);
    398        
    399         assertEquals("/c:CMD/c:Header/c:MdCollectionDisplayName/text()", 
    400                 mapping.getFallbackPatterns().get(0));
    401         assertEquals(1, mapping.getFallbackPatterns().size());
    402         mapping = facets.get(index++);
    403        
     393        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:LrtCommon/c:ResourceType/text()",
     394                mapping.getFallbackPatterns().get(0));
     395        mapping = facets.get(index++);
     396
     397        assertEquals("/c:CMD/c:Header/c:MdCollectionDisplayName/text()",
     398                mapping.getFallbackPatterns().get(0));
     399        assertEquals(1, mapping.getFallbackPatterns().size());
     400        mapping = facets.get(index++);
     401
    404402        assertEquals("/c:CMD/c:Header//text()", mapping.getFallbackPatterns().get(0));
    405403        assertEquals("/c:CMD/c:Components//text()", mapping.getFallbackPatterns().get(1));
    406404        assertEquals(2, mapping.getFallbackPatterns().size());
    407405        mapping = facets.get(index++);
    408        
     406
    409407        assertEquals("/c:CMD/c:Header/c:MdProfile/text()", mapping.getFallbackPatterns().get(0));
    410408        assertEquals(1, mapping.getFallbackPatterns().size());
    411409        mapping = facets.get(index++);
    412        
     410
    413411        assertEquals(FacetConstants.FIELD_KEYWORDS, mapping.getName());
    414412        assertEquals(1, mapping.getPatterns().size());
    415413        assertEquals(3, mapping.getFallbackPatterns().size());
    416         assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:tags/c:tag/text()", 
    417                 mapping.getPatterns().get(0));
    418         assertEquals("/c:CMD/c:Components/c:mods/c:classification/text()", 
     414        assertEquals("/c:CMD/c:Components/c:LrtInventoryResource/c:tags/c:tag/text()",
     415                mapping.getPatterns().get(0));
     416        assertEquals("/c:CMD/c:Components/c:mods/c:classification/text()",
    419417                mapping.getFallbackPatterns().get(0));
    420418        assertEquals("check to see we tested them all", facets.size(), index);
    … …  
    428426
    429427        List<FacetConfiguration> facets = facetMapping.getFacets();
    430        
     428
    431429        FacetConfiguration facet = facets.get(0);
    432        
     430
    433431        assertEquals(FacetConstants.FIELD_ID, facet.getName());
    434432        assertEquals(2, facet.getPatterns().size());
    435433        assertEquals("/c:CMD/c:Header/c:MdSelfLink/text()", facet.getPatterns().get(0));
    436         assertEquals("/c:CMD/c:Components/c:EastRepublican/c:GeneralInformation/c:Identifier/text()", 
     434        assertEquals("/c:CMD/c:Components/c:EastRepublican/c:GeneralInformation/c:Identifier/text()",
    437435                facet.getPatterns().get(1));
    438436    }
    439    
     437
    440438    /**
    441      * Tests black/white listing approach based on acceptableContext/rejectableContext information in facetconcepts.xml
    442      *
     439     * Tests black/white listing approach based on
     440     * acceptableContext/rejectableContext information in facetconcepts.xml
     441     *
    443442     * Not in use right now, therefore contains just a dummy test.
    444443     */
    445444    @Test
    446445    public void testConceptBasedBlacklisting() {
    447        assertTrue(true);
    448     }
    449    
     446        assertTrue(true);
     447    }
     448
    450449    /**
    451      * Tests black/white listing approach based on acceptableContext/rejectableContext information in configuration
     450     * Tests black/white listing approach based on
     451     * acceptableContext/rejectableContext information in configuration
    452452     */
    453453    @Test
    … …  
    456456                .getFacetMapping(FACETCONCEPTS_FILENAME, TEXTCORPUSPROFILE_PROFILE_ID, true);
    457457        List<FacetConfiguration> facets = facetMapping.getFacets();
    458        
     458
    459459        FacetConfiguration facet = facets.get(5);
    460460        assertEquals(FacetConstants.FIELD_TEMPORAL_COVERAGE, facet.getName());
    … …  
    462462        assertEquals("/c:CMD/c:Components/c:TextCorpusProfile/c:GeneralInfo/c:CompletionYear/text()", facet.getPatterns().get(0));
    463463        assertEquals("/c:CMD/c:Components/c:TextCorpusProfile/c:GeneralInfo/c:PublicationDate/text()", facet.getPatterns().get(1));
    464        
    465        
     464
    466465        facet = facets.get(13);
    467466        assertEquals(FacetConstants.FIELD_DESCRIPTION, facet.getName());
    468467        assertEquals(1, facet.getPatterns().size());
    469         assertEquals("/c:CMD/c:Components/c:TextCorpusProfile/c:GeneralInfo/c:Descriptions/c:Description/text()" ,facet.getPatterns().get(0));
     468        assertEquals("/c:CMD/c:Components/c:TextCorpusProfile/c:GeneralInfo/c:Descriptions/c:Description/text()", facet.getPatterns().get(0));
    470469    }
    471470}

  • vlo/branches/vlo-ticket761/vlo-importer/src/test/java/eu/clarin/cmdi/vlo/importer/ImporterTestcase.java

    r5997 r6267  
    77import java.io.File;
    88import java.io.IOException;
     9import java.net.URISyntaxException;
    910import org.apache.commons.io.FileUtils;
    1011import org.junit.After;
    … …  
    4041        MetadataImporter.config.setComponentRegistryRESTURL("http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/profiles/");
    4142        config = MetadataImporter.config;
    42        
     43
    4344        MetadataImporter.languageCodeUtils = new LanguageCodeUtils(config);
    4445    }
    4546
     47    public static String getTestFacetConceptFilePath() {
     48        try {
     49            return new File(ImporterTestcase.class.getResource("/facetConceptsTest.xml").toURI()).getAbsolutePath();
     50        } catch (URISyntaxException ex) {
     51            throw new RuntimeException(ex);
     52        }
     53    }
     54
    4655}

  • vlo/branches/vlo-ticket761/vlo-importer/src/test/java/eu/clarin/cmdi/vlo/importer/MetadataImporterTest.java

    r5979 r6267  
    4747        File sessionFile = createCmdiFile("testSession", session);
    4848
    49         String content = "";
    50         content += "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
    51         content += "<CMD xmlns=\"http://www.clarin.eu/cmd/\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://www.clarin.eu/cmd http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/profiles/clarin.eu:cr1:p_1274880881885/xsd\">\n";
    52         content += "   <Header>\n";
    53         content += "      <MdSelfLink>testID2</MdSelfLink>\n";
    54         content += "   </Header>\n";
    55         content += "   <Resources>\n";
    56         content += "      <ResourceProxyList>\n";
    57         content += "         <ResourceProxy id=\"d28635e19\">\n";
    58         content += "            <ResourceType>Metadata</ResourceType>\n";
    59         content += "            <ResourceRef>" + sessionFile.getName() + "</ResourceRef>\n";
    60         content += "         </ResourceProxy>\n";
    61         content += "      </ResourceProxyList>\n";
    62         content += "   </Resources>\n";
    63         content += "   <Components>\n";
    64         content += "      <imdi-corpus>\n";
    65         content += "         <Corpus>\n";
    66         content += "            <Name>MPI corpora</Name>\n";
    67         content += "         </Corpus>\n";
    68         content += "      </imdi-corpus>\n";
    69         content += "   </Components>\n";
    70         content += "</CMD>\n";
    71         File rootFile = createCmdiFile("rootFile", content);
    72 
    73         List<SolrInputDocument> docs = importData(rootFile.getParentFile());
     49        List<SolrInputDocument> docs = importData(sessionFile);
    7450        assertEquals(1, docs.size());
    7551        SolrInputDocument doc = docs.get(0);
    … …  
    261237               
    262238                // make sure the mapping file for testing is used
    263                 config.setFacetConceptsFile("/facetConceptsTest.xml");
     239                config.setFacetConceptsFile(getTestFacetConceptFilePath());
    264240               
    265241                List<DataRoot> dataRoots = checkDataRoots();
    … …  
    273249                                CMDIParserVTDXML(POST_PROCESSORS, true);
    274250                        List<File> files =
    275                                 getFilesFromDataRoot(dataRoot.getRootFile());
     251                                getFilesFromDataRoot(dataRoot.getRootFile()).get(0);
    276252                        for (File file : files) {
    277253                            if (config.getMaxFileSize () > 0

  • vlo/branches/vlo-ticket761/vlo-importer/src/test/java/eu/clarin/cmdi/vlo/importer/TemporalCoveragePostProcessorTest.java

    r6188 r6267  
    22
    33import static org.junit.Assert.assertEquals;
    4 import static org.junit.Assert.assertNull;
    54import org.junit.Test;
    65
    … …  
    1211        assertEquals("/2012", processor.process("/2012").get(0));
    1312        assertEquals("2012-02", processor.process("2012-02").get(0));
     13        assertEquals("1997-07-16/1997-07-17", processor.process("1997-07-16/1997-07-17").get(0));
     14        assertEquals("1994-11-05", processor.process("1994-11-05T08:15:30-05:00").get(0));
     15        assertEquals(0, processor.process("northlimit=-16.4933; southlimit=-16.5617; westlimit=167.419; eastlimit=167.46").size());
    1416        assertEquals(0, processor.process("1. November").size());
    1517    }

  • vlo/branches/vlo-ticket761/vlo-solr/pom.xml

    r5991 r6267  
    44        <groupId>eu.clarin.cmdi</groupId>
    55        <artifactId>vlo</artifactId>
    6         <version>3.2-SNAPSHOT</version>
     6        <version>3.3-SNAPSHOT</version>
    77    </parent>
    88   
    … …  
    1010    <groupId>eu.clarin.cmdi</groupId>
    1111    <artifactId>vlo-solr</artifactId>
    12     <version>3.2-SNAPSHOT</version>
     12    <version>3.3-SNAPSHOT</version>
    1313    <packaging>war</packaging>
    1414   

  • vlo/branches/vlo-ticket761/vlo-solr/src/main/solr_conf/solr/collection1/conf/schema.xml

    r6188 r6267  
    447447   <field name="_lastSeen" type="tdate" indexed="true" stored="true" multiValued="false" />
    448448   <field name="_searchPageRef" type="string" indexed="true" stored="true" multiValued="true" />
     449   <field name="_isPartOf" type="string" indexed="true" stored="true" multiValued="true" />
     450   <field name="_hasPart" type="string" indexed="true" stored="true" multiValued="true" />
     451   <field name="_hierarchyWeight" type="int" indexed="true" stored="true" multiValued="false" default='0'/>
    449452   <field name="metadataSource" type="metadataSource" indexed="false" stored="true" multiValued="true" />
     453   
     454   <!-- Field _version_ for update log (needed for partial updates) -->
     455   <field name="_version_" type="long" indexed="true" stored="true"/>
    450456   
    451457   <!-- Field for autocompletion (suggester) -->

  • vlo/branches/vlo-ticket761/vlo-web-app

    • Property svn:ignore

      •  

        old new  
        11target
         2nbactions-development_clone.xml

  • vlo/branches/vlo-ticket761/vlo-web-app/pom.xml

    r6002 r6267  
    77        <groupId>eu.clarin.cmdi</groupId>
    88        <artifactId>vlo</artifactId>
    9         <version>3.2-SNAPSHOT</version>
     9        <version>3.3-SNAPSHOT</version>
    1010    </parent>
    1111   
    … …  
    1414    <artifactId>vlo-web-app</artifactId>
    1515    <packaging>war</packaging>
    16     <version>3.2-SNAPSHOT</version>
     16    <version>3.3-SNAPSHOT</version>
    1717    <name>VLO Web Application</name>
    1818    <description></description>
    … …  
    2121        <wicket.version>6.19.0</wicket.version>
    2222        <spring.version>3.2.13.RELEASE</spring.version>
    23         <jetty.version>8.1.16.v20140903</jetty.version>
     23        <jetty.version>8.1.17.v20150415</jetty.version>
    2424        <!-- selects the appender for the log4j root logger -->
    2525        <log4j.appender>File</log4j.appender>
    … …  
    150150            <groupId>com.sun.jersey</groupId>
    151151            <artifactId>jersey-client</artifactId>
    152             <version>1.18.3</version>
     152            <version>1.19</version>
    153153        </dependency>
    154154        <dependency>
    155155            <groupId>org.apache.commons</groupId>
    156156            <artifactId>commons-lang3</artifactId>
    157             <version>3.3.2</version>
     157            <version>3.4</version>
    158158            <scope>test</scope>
    159159            <type>jar</type>
    … …  
    163163            <groupId>net.bull.javamelody</groupId>
    164164            <artifactId>javamelody-core</artifactId>
    165             <version>1.55.0</version>
     165            <version>1.56.0</version>
    166166            <exclusions>
    167167                <exclusion>
    … …  
    174174                </exclusion>
    175175            </exclusions>
     176        </dependency>
     177        <dependency>
     178            <groupId>nl.mpi.corpusstructure</groupId>
     179            <artifactId>corpus-structure-2-core</artifactId>
     180            <version>1.1.6</version>
    176181        </dependency>
    177182    </dependencies>
    … …  
    297302                <groupId>org.codehaus.mojo</groupId>
    298303                <artifactId>cobertura-maven-plugin</artifactId>
    299                 <version>2.6</version>
     304                <version>2.7</version>
    300305                <configuration>   
    301306                    <formats>
    … …  
    331336                <groupId>org.codehaus.mojo</groupId>
    332337                <artifactId>cobertura-maven-plugin</artifactId>
    333                 <version>2.6</version>
     338                <version>2.7</version>
    334339            </plugin>
    335340        </plugins>

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/VloWicketApplication.java

    r6046 r6267  
    11package eu.clarin.cmdi.vlo;
    22
     3import eu.clarin.cmdi.vlo.service.FacetDescriptionService;
    34import eu.clarin.cmdi.vlo.config.VloConfig;
     5import eu.clarin.cmdi.vlo.service.PermalinkService;
    46import eu.clarin.cmdi.vlo.service.XmlTransformationService;
    57import eu.clarin.cmdi.vlo.service.solr.SolrDocumentService;
    … …  
    4042    @Inject
    4143    private FieldValueConverterProvider fieldValueConverterProvider;
     44    @Inject
     45    private FacetDescriptionService facetDescriptionService;
     46    @Inject
     47    private PermalinkService permalinkService;
    4248    @Inject
    4349    private VloConfig vloConfig;
    … …  
    148154        return cmdiTransformationService;
    149155    }
    150    
     156
    151157    public FieldValueConverterProvider getFieldValueConverterProvider() {
    152158        return fieldValueConverterProvider;
    153159    }
    154160
     161    public FacetDescriptionService getFacetDescriptionService() {
     162        return facetDescriptionService;
     163    }
     164
     165    public PermalinkService getPermalinkService() {
     166        return permalinkService;
     167    }
     168
    155169}

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/config/VloServicesSpringConfig.java

    r6023 r6267  
    1919import com.google.common.collect.Sets;
    2020import eu.clarin.cmdi.vlo.LanguageCodeUtils;
     21import eu.clarin.cmdi.vlo.facets.FacetConceptsMarshaller;
    2122import eu.clarin.cmdi.vlo.pojo.QueryFacetsSelection;
    2223import eu.clarin.cmdi.vlo.pojo.SearchContext;
     24import eu.clarin.cmdi.vlo.service.FacetDescriptionService;
    2325import eu.clarin.cmdi.vlo.service.FacetParameterMapper;
    2426import eu.clarin.cmdi.vlo.service.FieldFilter;
    2527import eu.clarin.cmdi.vlo.service.PageParametersConverter;
     28import eu.clarin.cmdi.vlo.service.PermalinkService;
    2629import eu.clarin.cmdi.vlo.service.ResourceStringConverter;
    2730import eu.clarin.cmdi.vlo.service.ResourceTypeCountingService;
    2831import eu.clarin.cmdi.vlo.service.UriResolver;
    2932import eu.clarin.cmdi.vlo.service.XmlTransformationService;
    30 import eu.clarin.cmdi.vlo.service.handle.HandleClient;
    31 import eu.clarin.cmdi.vlo.service.handle.impl.HandleRestApiClient;
    3233import eu.clarin.cmdi.vlo.service.impl.DocumentParametersConverter;
    3334import eu.clarin.cmdi.vlo.service.impl.ExclusiveFieldFilter;
     35import eu.clarin.cmdi.vlo.service.impl.FacetDescriptionServiceImpl;
    3436import eu.clarin.cmdi.vlo.service.impl.FacetParameterMapperImpl;
    3537import eu.clarin.cmdi.vlo.service.impl.InclusiveFieldFilter;
     38import eu.clarin.cmdi.vlo.service.impl.PermalinkServiceImpl;
    3639import eu.clarin.cmdi.vlo.service.impl.QueryFacetsSelectionParametersConverter;
    3740import eu.clarin.cmdi.vlo.service.impl.ResourceStringConverterImpl;
    … …  
    4447import java.util.Properties;
    4548import javax.inject.Inject;
     49import javax.xml.bind.JAXBException;
    4650import javax.xml.transform.OutputKeys;
    4751import javax.xml.transform.Source;
    4852import javax.xml.transform.TransformerConfigurationException;
    4953import javax.xml.transform.stream.StreamSource;
     54import nl.mpi.archiving.corpusstructure.core.handle.CachingHandleResolver;
     55import nl.mpi.archiving.corpusstructure.core.handle.HandleApiResolver;
     56import nl.mpi.archiving.corpusstructure.core.handle.HandleResolver;
     57import nl.mpi.archiving.corpusstructure.core.handle.HttpHandleResolver;
    5058import org.apache.solr.common.SolrDocument;
    5159import org.springframework.context.annotation.Bean;
    … …  
    6169public class VloServicesSpringConfig {
    6270
     71    /**
     72     * Handle resolution cache expiry in seconds
     73     */
     74    private static final int HANDLE_CACHE_EXPIRY = 3600;
     75   
    6376    @Inject
    6477    VloConfig vloConfig;
    65 
     78   
    6679    @Bean
    6780    public ResourceTypeCountingService resourceTypeCountingService() {
    6881        return new ResourceTypeCountingServiceImpl(resourceStringConverter());
    6982    }
    70 
     83   
    7184    @Bean(name = "resourceStringConverter")
    7285    public ResourceStringConverter resourceStringConverter() {
    7386        return new ResourceStringConverterImpl();
    7487    }
    75 
     88   
    7689    @Bean(name = "resolvingResourceStringConverter")
    7790    public ResourceStringConverter resolvingResourceStringConverter() {
    7891        return new ResourceStringConverterImpl(uriResolver());
    7992    }
    80 
     93   
    8194    @Bean
    8295    public UriResolver uriResolver() {
    83         return new UriResolverImpl(handleClient());
     96        return new UriResolverImpl(handleResolver());
    8497    }
    85 
    86     public HandleClient handleClient() {
    87         return new HandleRestApiClient();
     98   
     99    public HandleResolver handleResolver() {
     100        return new CachingHandleResolver(new HttpHandleResolver(), HANDLE_CACHE_EXPIRY);
    88101    }
    89102   
    … …  
    92105        return new FacetParameterMapperImpl(languageCodeUtils());
    93106    }
    94 
     107   
    95108    @Bean(name = "queryParametersConverter")
    96109    public PageParametersConverter<QueryFacetsSelection> queryParametersConverter() {
    97110        return new QueryFacetsSelectionParametersConverter(vloConfig, facetParameterMapper());
    98111    }
    99 
     112   
    100113    @Bean(name = "documentParamsConverter")
    101114    public PageParametersConverter<SolrDocument> documentParamsConverter() {
    102115        return new DocumentParametersConverter();
    103116    }
    104 
     117   
    105118    @Bean(name = "searchContextParamsConverter")
    106119    public PageParametersConverter<SearchContext> searchContextParamsConverter() {
    107120        return new SearchContextParametersConverter(queryParametersConverter());
    108121    }
    109 
     122   
    110123    @Bean
    111124    public XmlTransformationService cmdiTransformationService() throws TransformerConfigurationException {
    … …  
    118131        return new XmlTransformationServiceImpl(xsltSource, transformationProperties);
    119132    }
    120 
     133   
    121134    @Bean(name = "basicPropertiesFilter")
    122135    public FieldFilter basicPropertiesFieldFilter() {
    … …  
    125138                vloConfig.getTechnicalFields()));
    126139    }
    127 
     140   
    128141    @Bean(name = "searchResultPropertiesFilter")
    129142    public FieldFilter searchResultPropertiesFilter() {
    130143        return new InclusiveFieldFilter(vloConfig.getSearchResultFields());
    131144    }
    132 
     145   
    133146    @Bean(name = "technicalPropertiesFilter")
    134147    public FieldFilter technicalPropertiesFieldFilter() {
    … …  
    136149                vloConfig.getTechnicalFields());
    137150    }
    138 
     151   
    139152    @Bean
    140153    public LanguageCodeUtils languageCodeUtils() {
    141154        return new LanguageCodeUtils(vloConfig);
    142155    }
    143 
     156   
    144157    @Bean
    145158    public FieldValueConverterProvider fieldValueConverters() {
    146159        return new FieldValueConverterProviderImpl(languageCodeUtils());
    147160    }
    148 
     161   
     162    @Bean
     163    public FacetDescriptionService facetDescriptionsService() throws JAXBException {
     164        return new FacetDescriptionServiceImpl(facetConceptsMarshaller(), vloConfig);
     165    }
     166   
     167    @Bean
     168    public FacetConceptsMarshaller facetConceptsMarshaller() throws JAXBException {
     169        return new FacetConceptsMarshaller();
     170    }
     171   
     172    @Bean
     173    public PermalinkService permalinkService() {
     174        return new PermalinkServiceImpl(queryParametersConverter());
     175    }
     176   
    149177}

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/config/VloSolrSpringConfig.java

    r6087 r6267  
    118118            FacetConstants.FIELD_LAST_SEEN,
    119119            FacetConstants.FIELD_CLARIN_PROFILE,
    120             FacetConstants.FIELD_COMPLETE_METADATA
     120            FacetConstants.FIELD_COMPLETE_METADATA,
     121            FacetConstants.FIELD_SOLR_SCORE
    121122    );
    122123}

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/service/impl/UriResolverImpl.java

    r5027 r6267  
    11/*
    2  * Copyright (C) 2014 CLARIN
     2 * Copyright (C) 2015 CLARIN
    33 *
    44 * This program is free software: you can redistribute it and/or modify
    … …  
    1717package eu.clarin.cmdi.vlo.service.impl;
    1818
    19 import eu.clarin.cmdi.vlo.service.handle.impl.HandleRestApiClient;
    2019import static eu.clarin.cmdi.vlo.FacetConstants.HANDLE_PREFIX;
    2120import static eu.clarin.cmdi.vlo.FacetConstants.HANDLE_PROXY;
    22 import eu.clarin.cmdi.vlo.service.handle.HandleClient;
    2321import eu.clarin.cmdi.vlo.service.UriResolver;
     22import java.net.URI;
     23import java.net.URISyntaxException;
     24import nl.mpi.archiving.corpusstructure.core.handle.HandleResolver;
     25import nl.mpi.archiving.corpusstructure.core.handle.InvalidHandleException;
    2426import org.slf4j.Logger;
    2527import org.slf4j.LoggerFactory;
    … …  
    2830 * Resolves a URI as follows: if the URI starts with the handle scheme or the
    2931 * handle proxy, the handle is extracted and passed on to this resolver's
    30  * {@link HandleClient} and the result of {@link HandleClient#getUrl(java.lang.String)
    31  * } is returned; otherwise the original URI is returned.
     32 * {@link HandleResolver} and the result of {@link HandleResolver#resolve(java.net.URI)
     33 * } is returned (as String); otherwise the original URI is returned.
    3234 *
    33  * TODO: add support for resolving URN:NBN <https://trac.clarin.eu/ticket/535>
    34  *
    35  * @author twagoo
     35 * @author Twan Goosen <twan.goosen@mpi.nl>
    3636 */
    3737public class UriResolverImpl implements UriResolver {
    3838
    39     private final static Logger logger = LoggerFactory.getLogger(HandleRestApiClient.class);
     39    private final static Logger logger = LoggerFactory.getLogger(UriResolverImpl.class);
    4040
    41     private final HandleClient handleClient;
     41    private final HandleResolver handleClient;
    4242
    43     public UriResolverImpl(HandleClient handleClient) {
     43    public UriResolverImpl(HandleResolver handleClient) {
    4444        this.handleClient = handleClient;
    4545    }
    … …  
    4949        final String handle = getHandle(uri);
    5050
    51         if (handle == null) {
    52             return uri;
    53         } else {
     51        if (handle != null) {
    5452            logger.debug("Calling handle client to resolve handle [{}]", uri);
    55             final String resolved = handleClient.getUrl(handle);
    56             if (resolved == null) {
    57                 return uri;
    58             } else {
    59                 return resolved;
     53            try {
     54                final URI resolved = handleClient.resolve(new URI(handle));
     55                if (resolved != null) {
     56                    return resolved.toString();
     57                }
     58            } catch (InvalidHandleException ex) {
     59                logger.warn("Invalid handle ecountered: {}", handle);
     60            } catch (URISyntaxException ex) {
     61                logger.warn("Invalid URI for handle: {}", handle);
    6062            }
    6163        }
     64        // not a resolvable handle
     65        return uri;
    6266
    6367    }
    6468
    6569    private String getHandle(String uri) {
    66         final String handle;
    6770        if (uri.startsWith(HANDLE_PREFIX)) {
    68             handle = uri.substring(HANDLE_PREFIX.length());
     71            return uri;
    6972        } else if (uri.startsWith(HANDLE_PROXY)) {
    70             handle = uri.substring(HANDLE_PROXY.length());
     73            return HANDLE_PREFIX + uri.substring(HANDLE_PROXY.length());
    7174        } else {
    72             handle = null;
     75            return null;
    7376        }
    74         return handle;
    7577    }
    7678

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/components/LanguageInfoLink.java

    r6112 r6267  
    1919import eu.clarin.cmdi.vlo.FacetConstants;
    2020import eu.clarin.cmdi.vlo.config.VloConfig;
     21import static eu.clarin.cmdi.vlo.config.VloConfig.LANGUAGE_LINK_TEMPLATE_LANGUAGE_CODE_PLACEHOLDER;
    2122import java.util.regex.Matcher;
    2223import java.util.regex.Pattern;
    … …  
    3940 * @author Twan Goosen <twan.goosen@mpi.nl>
    4041 * @see FacetConstants#FIELD_LANGUAGE_CODE
    41  * @see VloConfig#getLanguageLinkPrefix()
     42 * @see VloConfig#getLanguageLinkTemplate()
    4243 */
    4344public class LanguageInfoLink extends GenericPanel<String> {
    … …  
    6465                    return null;
    6566                } else {
    66                     return vloConfig.getLanguageLinkPrefix() + languageCode;
     67                    return vloConfig.getLanguageLinkTemplate().replace(LANGUAGE_LINK_TEMPLATE_LANGUAGE_CODE_PLACEHOLDER, languageCode);
    6768                }
    6869            }

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/model/FacetFieldSelectionModel.java

    r4963 r6267  
    8686    }
    8787
     88    @Override
     89    public String toString() {
     90        return String.format("[Field: %s; Selection: %s]", facetFieldModel.getObject().getName(), selectionModel.getObject().toString());
     91    }
     92   
    8893}

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/model/SolrFieldNameModel.java

    r4663 r6267  
    1717package eu.clarin.cmdi.vlo.wicket.model;
    1818
    19 import eu.clarin.cmdi.vlo.pojo.DocumentField;
    2019import org.apache.solr.client.solrj.response.FacetField;
    2120import org.apache.wicket.Application;

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/pages/AboutPage.html

    r5912 r6267  
    7575                <h3>Publications</h3>
    7676                <ul>
    77                     <li><a href="http://hdl.handle.net/11858/00-001M-0000-000F-85F4-5">Van Uytvanck, D., Stehouwer, H., & Lampen, L. (2012). Semantic metadata mapping in practice: The Virtual Language Observatory. In N. Calzolari (Ed.), Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC 2012), Istanbul, May 23rd-25th, 2012 (pp. 1029-1034). European Language Resources Association (ELRA).</a></li>
     77                    <li><a href="http://hdl.handle.net/11858/00-001M-0000-000F-85F4-5">Van Uytvanck, D., Stehouwer, H., &amp; Lampen, L. (2012). Semantic metadata mapping in practice: The Virtual Language Observatory. In N. Calzolari (Ed.), Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC 2012), Istanbul, May 23rd-25th, 2012 (pp. 1029-1034). European Language Resources Association (ELRA).</a></li>
     78                    <li><a href="http://www.clarin.eu/sites/default/files/cac2014_submission_2_0.pdf">Goosen, T., &amp; Eckart, T. (2014). Virtual Language Observatory 3.0: What’s New? CLARIN Annual Conference 2014, Soesterberg, Netherlands.</a></li>
    7879                </ul>
    7980

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/pages/FacetedSearchPage.java

    r5205 r6267  
    11package eu.clarin.cmdi.vlo.wicket.pages;
    22
     3import eu.clarin.cmdi.vlo.wicket.model.PermaLinkModel;
    34import eu.clarin.cmdi.vlo.wicket.panels.SingleFacetPanel;
    45import eu.clarin.cmdi.vlo.config.VloConfig;
    … …  
    4647    private Component collectionsPanel;
    4748    private Component navigation;
     49    private Component searchForm;
    4850
    4951    public FacetedSearchPage(IModel<QueryFacetsSelection> queryModel) {
    … …  
    6466        navigation = createNavigation("navigation");
    6567        add(navigation);
    66 
    67         add(createSearchForm("search"));
     68       
     69        searchForm = createSearchForm("search");
     70        add(searchForm);
    6871
    6972        collectionsPanel = createCollectionsPanel("collections");
    … …  
    9295
    9396        });
    94         container.add(new TopLinksPanel("permalink", getModel()) {
     97        container.add(new TopLinksPanel("permalink", new PermaLinkModel(getPageClass(), getModel())) {
    9598
    9699            @Override
    … …  
    117120
    118121    private SearchForm createSearchForm(String id) {
    119         final SearchForm searchForm = new SearchForm(id, getModel()) {
     122        final SearchForm form = new SearchForm(id, getModel()) {
    120123
    121124            @Override
    … …  
    127130
    128131        };
    129         return searchForm;
     132        form.setOutputMarkupId(true);
     133        return form;
    130134    }
    131135
    … …  
    171175        if (target != null) { // null if JavaScript disabled
    172176            target.add(navigation);
     177            target.add(searchForm);
    173178            target.add(searchResultsPanel);
    174179            target.add(facetsPanel);
    … …  
    176181        }
    177182    }
     183
     184    @Override
     185    public IModel<String> getCanonicalUrlModel() {
     186        return new PermaLinkModel(getPageClass(), getModel());
     187    }
    178188}

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/pages/RecordPage.java

    r5315 r6267  
    1717package eu.clarin.cmdi.vlo.wicket.pages;
    1818
     19import eu.clarin.cmdi.vlo.wicket.model.PermaLinkModel;
    1920import eu.clarin.cmdi.vlo.FacetConstants;
    2021import eu.clarin.cmdi.vlo.VloWebAppParameters;
    … …  
    177178
    178179    private TopLinksPanel createPermalink(String id, final WebMarkupContainer topNavigation) {
    179         return new TopLinksPanel(id, selectionModel, getModel()) {
     180        return new TopLinksPanel(id, new PermaLinkModel(getPageClass(), selectionModel, getModel())) {
    180181
    181182            @Override
    … …  
    293294
    294295    @Override
     296    public IModel<String> getCanonicalUrlModel() {
     297        // omit query in link for canonical URL (record page gets same canonical
     298        // URL regardless of search term)
     299        return new PermaLinkModel(getPageClass(), null, getModel());
     300    }
     301
     302    @Override
    295303    public void renderHead(IHeaderResponse response) {
    296304        super.renderHead(response);

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/pages/SimpleSearchPage.java

    r5205 r6267  
    6969
    7070        navigation.add(new BookmarkablePageLink("breadcrumb", getApplication().getHomePage()));
    71         navigation.add(new TopLinksPanel("topLinks"));
     71        navigation.add(new TopLinksPanel("topLinks", null));
    7272
    7373        // add a persistenet panel for selection of a value for the collection facet

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/pages/VloBasePage.html

    r6160 r6267  
    3131        <meta name="viewport" content="width=device-width"/>
    3232        <meta wicket:id="pageDescription" name="description" content="" />
     33        <link wicket:id="canonicalUrl" rel="canonical" />
    3334    </head>
    3435

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/pages/VloBasePage.java

    r5315 r6267  
    1717package eu.clarin.cmdi.vlo.wicket.pages;
    1818
    19 import com.google.common.base.Strings;
    2019import eu.clarin.cmdi.vlo.JavaScriptResources;
    2120import eu.clarin.cmdi.vlo.VloWebAppParameters;
    … …  
    4140import org.slf4j.Logger;
    4241import org.slf4j.LoggerFactory;
    43 import org.springframework.util.StringUtils;
    4442
    4543/**
    … …  
    124122            }
    125123        });
     124
     125        add(new WebComponent("canonicalUrl") {
     126
     127            @Override
     128            protected void onRender() {
     129                final IModel<String> canonicalUrlModel = getCanonicalUrlModel();
     130                if (canonicalUrlModel != null) {
     131                    getResponse().write("<link rel=\"canonical\" href=\"" + canonicalUrlModel.getObject() + "\"/>");
     132                }
     133            }
     134
     135        });
    126136    }
    127137
    … …  
    146156    }
    147157
     158    /**
     159     *
     160     * @return URL to include as a canonical HREF in the page header (null to
     161     * omit such a reference)
     162     */
     163    public IModel<String> getCanonicalUrlModel() {
     164        return null;
     165    }
     166
    148167    @Override
    149168    public void renderHead(IHeaderResponse response) {
    … …  
    161180    }
    162181
     182
    163183}

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/BreadCrumbPanel.html

    r5224 r6267  
    3030            <wicket:container wicket:id="query">
    3131                &gt; <a wicket:id="leavequery">Search: &quot;<span wicket:id="content"></span>&quot;</a>
     32                <a wicket:id="removal" title="Remove query" class="breadcrumbunselect"><span>[Remove]</span></a>
    3233            </wicket:container>
    3334            <wicket:container wicket:id="facets">
    3435                &gt; <a wicket:id="leaveselection">Selections</a>:
    3536                <wicket:container wicket:id="facet">
    36                     <span class="breadcrumbfacet"><wicket:container wicket:id="value">[VALUE]</wicket:container> <a wicket:id="removal" title="Remove selection" class="breadcrumbunselect"><span>[Remove]</span></a></span>
     37                    <span class="breadcrumbfacet"><wicket:container wicket:id="value">[VALUE]</wicket:container>
     38                        <a wicket:id="removal" title="Remove selection" class="breadcrumbunselect"><span>[Remove]</span></a></span>
    3739                </wicket:container>
    3840            </wicket:container>

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/BreadCrumbPanel.java

    r6012 r6267  
    5353 */
    5454public class BreadCrumbPanel extends GenericPanel<QueryFacetsSelection> {
    55    
     55
    5656    @SpringBean(name = "queryParametersConverter")
    5757    private PageParametersConverter<QueryFacetsSelection> paramsConverter;
    5858    @SpringBean
    5959    private FieldValueConverterProvider fieldValueConverterProvider;
    60    
     60
    6161    private final WebMarkupContainer query;
    6262    private final WebMarkupContainer facets;
    63    
     63
    6464    public BreadCrumbPanel(String id, IModel<QueryFacetsSelection> model) {
    6565        super(id, model);
    … …  
    6868        add(facets = createFacets(model, "facets"));
    6969    }
    70    
     70
    7171    private WebMarkupContainer createQuery(final IModel<QueryFacetsSelection> selectionModel, String id) {
    7272        final WebMarkupContainer queryContainer = new WebMarkupContainer(id);
    7373        final Link link = new AjaxFallbackLink("leavequery") {
    74            
     74
    7575            @Override
    7676            public void onClick(AjaxRequestTarget target) {
    … …  
    8282        link.add(new Label("content", new PropertyModel(selectionModel, "query")));
    8383        queryContainer.add(link);
     84
     85        queryContainer.add(new AjaxFallbackLink("removal") {
     86
     87            @Override
     88            public void onClick(AjaxRequestTarget target) {
     89                // get a copy of the current selection
     90                final QueryFacetsSelection newSelection = selectionModel.getObject().getCopy();
     91                newSelection.setQuery(null);
     92                onSelectionChanged(newSelection, target);
     93            }
     94        });
    8495        return queryContainer;
    8596    }
    86    
     97
    8798    private WebMarkupContainer createFacets(final IModel<QueryFacetsSelection> model, String id) {
    8899        final WebMarkupContainer facetsContainer = new WebMarkupContainer(id);
    89100        facetsContainer.add(new AjaxFallbackLink("leaveselection") {
    90            
     101
    91102            @Override
    92103            public void onClick(AjaxRequestTarget target) {
    … …  
    98109        final FacetSelectionProvider facetSelectionProvider = new FacetSelectionProvider(model);
    99110        facetsContainer.add(new DataView<Map.Entry<String, FacetSelection>>("facet", facetSelectionProvider) {
    100            
     111
    101112            @Override
    102113            protected void populateItem(final Item<Map.Entry<String, FacetSelection>> item) {
    … …  
    104115                // add a label for the selected facet value(s)
    105116                final Label valueLabel = new Label("value", new PropertyModel(selectionModel, "value")) {
    106                    
     117
    107118                    @Override
    108119                    public <C> IConverter<C> getConverter(Class<C> type) {
    … …  
    111122                        return (IConverter<C>) new SelectionConverter(facet, fieldValueConverterProvider.getConverter(facet));
    112123                    }
    113                    
     124
    114125                };
    115126                // add facet name as title attribute so that it becomes available through a tooltip
    … …  
    120131                // add a link for removal of the facet value selection
    121132                item.add(new AjaxFallbackLink("removal") {
    122                    
     133
    123134                    @Override
    124135                    public void onClick(AjaxRequestTarget target) {
    … …  
    133144            }
    134145        });
    135        
     146
    136147        return facetsContainer;
    137148    }
    … …  
    148159        setResponsePage(FacetedSearchPage.class, paramsConverter.toParameters(selection));
    149160    }
    150    
     161
    151162    @Override
    152163    protected void onConfigure() {
    153164        super.onConfigure();
    154        
     165
    155166        final String queryString = getModelObject().getQuery();
    156167        final Map<String, FacetSelection> selection = getModelObject().getSelection();
    157        
     168
    158169        query.setVisible(queryString != null && !queryString.isEmpty());
    159170        facets.setVisible(selection != null && !selection.isEmpty());
    … …  
    165176     */
    166177    private class SelectionConverter implements IConverter<FacetSelection> {
    167        
     178
    168179        private final String facet;
    169180        private final IConverter<String> valueConverter;
    170        
     181
    171182        public SelectionConverter(String facet, IConverter<String> valueConverter) {
    172183            this.facet = facet;
    173184            this.valueConverter = valueConverter;
    174185        }
    175        
     186
    176187        @Override
    177188        public FacetSelection convertToObject(String value, Locale locale) throws ConversionException {
    178189            throw new UnsupportedOperationException("Not supported yet.");
    179190        }
    180        
     191
    181192        @Override
    182193        public String convertToString(FacetSelection selection, Locale locale) {
    … …  
    193204                    return facet;
    194205            }
    195            
    196         }
    197        
     206
     207        }
     208
    198209        private String getAnyValueString() {
    199210            if (FacetConstants.FIELD_SEARCH_SERVICE.equals(facet)) {
    … …  
    202213            return "any " + facet;
    203214        }
    204        
     215
    205216        public String getCollectionString(FacetSelection selection, String valueSeparator, Locale locale) {
    206217            final Collection<String> value = selection.getValues();
    … …  
    219230            }
    220231        }
    221        
     232
    222233        private String getConvertedValue(String string, Locale locale) {
    223234            if (valueConverter != null) {
    … …  
    229240            return string;
    230241        }
    231        
     242
    232243    };
    233244}

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/ExpandablePanel.html

    r4976 r6267  
    2626            <a wicket:id="expand" class="expandfacet"><span>expand</span></a>
    2727            <a wicket:id="collapse" class="collapsefacet"><span>collapse</span></a>
    28             <h1><a wicket:id="titleToggle" href="#"><wicket:container wicket:id="title">[TITLE]</wicket:container></a></h1>
     28            <h1><a wicket:id="titleToggle" href="#"><span wicket:id="title">[TITLE]</span></a></h1>
    2929            <wicket:child />
    3030        </wicket:panel>

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/TopLinksPanel.java

    r5135 r6267  
    1717package eu.clarin.cmdi.vlo.wicket.panels;
    1818
    19 import eu.clarin.cmdi.vlo.FacetConstants;
    20 import eu.clarin.cmdi.vlo.VloWebAppParameters;
    2119import eu.clarin.cmdi.vlo.config.VloConfig;
    22 import eu.clarin.cmdi.vlo.pojo.QueryFacetsSelection;
    23 import eu.clarin.cmdi.vlo.service.PageParametersConverter;
    24 import org.apache.solr.common.SolrDocument;
    2520import org.apache.wicket.Component;
    26 import org.apache.wicket.Session;
    2721import org.apache.wicket.ajax.AjaxRequestTarget;
    2822import org.apache.wicket.ajax.markup.html.form.AjaxFallbackButton;
    29 import org.apache.wicket.extensions.ajax.markup.html.IndicatingAjaxFallbackLink;
    3023import org.apache.wicket.markup.html.form.Form;
    3124import org.apache.wicket.markup.html.form.TextField;
    … …  
    3326import org.apache.wicket.markup.html.link.Link;
    3427import org.apache.wicket.markup.html.panel.GenericPanel;
    35 import org.apache.wicket.model.AbstractReadOnlyModel;
    3628import org.apache.wicket.model.IModel;
    3729import org.apache.wicket.model.Model;
    38 import org.apache.wicket.request.Url;
    39 import org.apache.wicket.request.cycle.RequestCycle;
    4030import org.apache.wicket.request.http.handler.RedirectRequestHandler;
    41 import org.apache.wicket.request.mapper.parameter.PageParameters;
    4231import org.apache.wicket.spring.injection.annot.SpringBean;
    4332import org.apache.wicket.util.encoding.UrlEncoder;
     33import org.slf4j.LoggerFactory;
    4434
    4535/**
    … …  
    5646 * @author twagoo
    5747 */
    58 public class TopLinksPanel extends GenericPanel<QueryFacetsSelection> {
     48public class TopLinksPanel extends GenericPanel<String> {
    5949
    60     @SpringBean(name = "queryParametersConverter")
    61     private PageParametersConverter<QueryFacetsSelection> paramsConverter;
    6250    @SpringBean
    6351    private VloConfig vloConfig;
    6452
    65     private final IModel<SolrDocument> documentModel;
    6653    private final Model<Boolean> linkVisibilityModel;
    6754
    68     public TopLinksPanel(String id) {
    69         this(id, null, null);
    70     }
    71 
    72     public TopLinksPanel(String id, final IModel<QueryFacetsSelection> selectionmodel) {
    73         this(id, selectionmodel, null);
    74     }
    75 
    76     public TopLinksPanel(String id, final IModel<QueryFacetsSelection> selectionmodel, final IModel<SolrDocument> documentModel) {
    77         super(id, selectionmodel);
    78         this.documentModel = documentModel;
    79         this.linkVisibilityModel = new Model<Boolean>(false);
    80 
    81         // create a model that provides a link to the current page
    82         final IModel<String> linkModel = new PermaLinkModel(selectionmodel, documentModel);
     55    public TopLinksPanel(String id, final IModel<String> linkModel) {
     56        super(id, linkModel);
     57        this.linkVisibilityModel = new Model<>(false);
    8358
    8459        // action to link to request the permalink
    … …  
    11085            @Override
    11186            protected void onConfigure() {
    112                 super.onConfigure();
    11387                setVisible(TopLinksPanel.this.getModel() != null);
    11488            }
    … …  
    137111    private TextField<String> createLinkField(String id, final IModel<String> linkModel) {
    138112        final TextField<String> linkField = new TextField<String>(id, linkModel) {
     113
    139114            @Override
    140115            protected void onConfigure() {
    141                 super.onConfigure();
    142116                setVisible(linkVisibilityModel.getObject());
    143117            }
    … …  
    145119        };
    146120        return linkField;
    147     }
    148 
    149     @Override
    150     public void detachModels() {
    151         super.detachModels();
    152         if (documentModel != null) {
    153             documentModel.detach();
    154         }
    155121    }
    156122
    … …  
    161127    }
    162128
    163     private class PermaLinkModel extends AbstractReadOnlyModel<String> {
    164 
    165         private final IModel<QueryFacetsSelection> selectionmodel;
    166         private final IModel<SolrDocument> documentModel;
    167 
    168         public PermaLinkModel(IModel<QueryFacetsSelection> selectionmodel, IModel<SolrDocument> documentModel) {
    169             this.selectionmodel = selectionmodel;
    170             this.documentModel = documentModel;
    171         }
    172 
    173         @Override
    174         public String getObject() {
    175             final PageParameters params = new PageParameters();
    176             if (selectionmodel != null) {
    177                 params.mergeWith(paramsConverter.toParameters(selectionmodel.getObject()));
    178             }
    179 
    180             if (documentModel != null) {
    181                 params.add(VloWebAppParameters.DOCUMENT_ID, documentModel.getObject().getFirstValue(FacetConstants.FIELD_ID));
    182             }
    183 
    184             final String style = Session.get().getStyle();
    185             if (style != null) {
    186                 params.add(VloWebAppParameters.THEME, style);
    187             }
    188 
    189             final CharSequence url = urlFor(getPage().getClass(), params);
    190             final String absoluteUrl = RequestCycle.get().getUrlRenderer().renderFullUrl(Url.parse(url));
    191             return absoluteUrl;
    192         }
     129    @Override
     130    protected void onConfigure() {
     131        LoggerFactory.getLogger(getClass()).debug("top links panel onconfigure");
    193132    }
     133   
     134   
    194135
    195136}

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/record/ResourceLinksPanel.html

    r4904 r6267  
    2525        <wicket:panel>
    2626            <h2>Resources</h2>
    27             <wicket:container wicket:id="resource">
    28                 <a class="resourceLink" wicket:id="showResource">
    29                     <wicket:container wicket:id="filename">resource.txt</wicket:container>
    30                     <span wicket:id="details" class="recordresourcedetails"></span>
    31                 </a>
    32             </wicket:container>
     27            <div id="resourceList">
     28                <wicket:container wicket:id="resource">
     29                    <a class="resourceLink" wicket:id="showResource">
     30                        <wicket:container wicket:id="filename">resource.txt</wicket:container>
     31                        <span wicket:id="details" class="recordresourcedetails"></span>
     32                    </a>
     33                </wicket:container>
     34            </div>
     35            <div wicket:id="paging" id="resourcespagination" class="resultpagination">[paging]</div>
    3336        </wicket:panel>
    3437    </body>

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/record/ResourceLinksPanel.java

    r5165 r6267  
    2626import java.util.List;
    2727import org.apache.wicket.ajax.AjaxRequestTarget;
     28import org.apache.wicket.ajax.markup.html.navigation.paging.AjaxPagingNavigator;
    2829import org.apache.wicket.markup.html.basic.Label;
    2930import org.apache.wicket.markup.html.link.ExternalLink;
    3031import org.apache.wicket.markup.html.list.ListItem;
    31 import org.apache.wicket.markup.html.list.ListView;
     32import org.apache.wicket.markup.html.list.PageableListView;
    3233import org.apache.wicket.markup.html.panel.Panel;
    3334import org.apache.wicket.model.IModel;
    … …  
    4344public class ResourceLinksPanel extends Panel {
    4445
     46    private static final int ITEMS_PER_PAGE = 12;
     47
    4548    @SpringBean(name = "resourceStringConverter")
    4649    private ResourceStringConverter resourceStringConverter;
    … …  
    5558    public ResourceLinksPanel(String id, IModel<Collection<String>> model) {
    5659        super(id, model);
     60
    5761        // list view that shows all resources as links that show a resource details panel when clicked
    58         add(new ResourcesListView("resource", new CollectionListModel<String>(model)));
     62        final ResourcesListView resourceListing = new ResourcesListView("resource", new CollectionListModel<String>(model));
     63        add(resourceListing);
     64
     65        // pagination
     66        add(new AjaxPagingNavigator("paging", resourceListing) {
     67
     68            @Override
     69            protected void onConfigure() {
     70                setVisible(resourceListing.getPageCount() > 1);
     71            }
     72
     73        });
     74
     75        //For Ajax updating of resource listing when paging
     76        setOutputMarkupId(true);
    5977    }
    6078
    61     private class ResourcesListView extends ListView<String> {
     79    private class ResourcesListView extends PageableListView<String> {
    6280
    6381        public ResourcesListView(String id, IModel<? extends List<? extends String>> model) {
    64             super(id, model);
     82            super(id, model, ITEMS_PER_PAGE);
    6583            setReuseItems(true);
    6684        }
    … …  
    6987        protected void populateItem(ListItem<String> item) {
    7088            final ResourceInfoModel resourceInfoModel = new ResourceInfoModel(resourceStringConverter, item.getModel());
    71            
     89
    7290            // add a link that will show the resource details panel when clicked
    73            
    7491            // wrap href in model that transforms handle links
    7592            final IModel<String> linkModel = new HandleLinkModel(new PropertyModel(resourceInfoModel, "href"));

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/search/FacetPanel.java

    r5140 r6267  
    1717package eu.clarin.cmdi.vlo.wicket.panels.search;
    1818
     19import eu.clarin.cmdi.vlo.wicket.model.SolrFieldDescriptionModel;
    1920import eu.clarin.cmdi.vlo.pojo.ExpansionState;
    2021import eu.clarin.cmdi.vlo.pojo.FacetFieldSelection;
    … …  
    2728import org.apache.solr.client.solrj.response.FacetField;
    2829import org.apache.wicket.ajax.AjaxRequestTarget;
     30import org.apache.wicket.behavior.AttributeAppender;
    2931import org.apache.wicket.markup.html.basic.Label;
    3032import org.apache.wicket.model.IModel;
    3133import org.apache.wicket.model.PropertyModel;
     34import org.slf4j.Logger;
     35import org.slf4j.LoggerFactory;
    3236
    3337/**
    … …  
    4044 */
    4145public abstract class FacetPanel extends ExpandablePanel<FacetFieldSelection> {
     46    private final static Logger logger = LoggerFactory.getLogger(FacetPanel.class);
    4247
    4348    private final IModel<ExpansionState> expansionStateModel;
    … …  
    6570    @Override
    6671    protected Label createTitleLabel(String id) {
    67         return new Label(id, new SolrFieldNameModel(new PropertyModel(getModel(), "facetField.name")));
     72        final IModel<String> facetNameModel = new PropertyModel<>(getModel(), "facetField.name");
     73        final Label label = new Label(id, new SolrFieldNameModel(facetNameModel));
     74        label.add(new AttributeAppender("title", new SolrFieldDescriptionModel(facetNameModel)));
     75        return label;
    6876    }
    6977

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/search/SearchResultItemPanel.html

    r4937 r6267  
    2424    <body>
    2525        <wicket:panel>
     26            <div class="score" wicket:id="scoreContainer">
     27                Score:  <span wicket:id="score">[score]</span>
     28            </div>
    2629            <h3>
    2730                <a wicket:id="recordLink" href="record1.html"><wicket:container wicket:id="title" >[TITLE]</wicket:container></a>

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/search/SearchResultItemPanel.java

    r4998 r6267  
    1818
    1919import eu.clarin.cmdi.vlo.FacetConstants;
     20import eu.clarin.cmdi.vlo.config.VloConfig;
    2021import eu.clarin.cmdi.vlo.pojo.ExpansionState;
    2122import eu.clarin.cmdi.vlo.pojo.SearchContext;
    2223import eu.clarin.cmdi.vlo.wicket.components.RecordPageLink;
    2324import eu.clarin.cmdi.vlo.wicket.components.SolrFieldLabel;
     25import eu.clarin.cmdi.vlo.wicket.model.SolrFieldStringModel;
    2426import org.apache.solr.common.SolrDocument;
     27import org.apache.wicket.MarkupContainer;
    2528import org.apache.wicket.ajax.AjaxRequestTarget;
    2629import org.apache.wicket.extensions.ajax.markup.html.IndicatingAjaxFallbackLink;
     30import org.apache.wicket.markup.html.WebMarkupContainer;
    2731import org.apache.wicket.markup.html.basic.Label;
    2832import org.apache.wicket.markup.html.link.Link;
    … …  
    3034import org.apache.wicket.model.AbstractReadOnlyModel;
    3135import org.apache.wicket.model.IModel;
     36import org.apache.wicket.spring.injection.annot.SpringBean;
    3237
    3338/**
    … …  
    3742public class SearchResultItemPanel extends Panel {
    3843
     44    @SpringBean
     45    private VloConfig config;
     46   
    3947    private final Panel collapsedDetails;
    4048    private final Panel expandedDetails;
    … …  
    6876        expandedDetails = new SearchResultItemExpandedPanel("expandedDetails", documentModel, selectionModel);
    6977        add(expandedDetails);
     78
     79        final MarkupContainer scoreContainer = new WebMarkupContainer("scoreContainer");
     80        scoreContainer.add(new Label("score", new SolrFieldStringModel(documentModel, FacetConstants.FIELD_SOLR_SCORE)));
     81        scoreContainer.setVisible(config.isShowResultScores());
     82        add(scoreContainer);
    7083
    7184        setOutputMarkupId(true);

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/search/SearchResultsPanel.html

    r5352 r6267  
    2929                <span wicket:id="resultCount" id="searchresultcount">604056 results</span>
    3030                <span wicket:id="resultPageIndicator" id="searchresultpagecount">Showing 1 to 25</span>
    31                 <div class="searchresultpagination">
     31                <div class="resultpagination">
    3232                    <div wicket:id="pagingTop">[PAGING]</div>
    3333                </div>
    … …  
    4040            </ul>
    4141           
    42             <div id="bottompagination" class="searchresultpagination">
     42            <div id="bottompagination" class="resultpagination">
    4343                <div wicket:id="pagingBottom">[PAGING]</div>
    4444

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/search/SearchResultsPanel.java

    r5353 r6267  
    4141import org.apache.wicket.model.Model;
    4242import org.apache.wicket.model.PropertyModel;
    43 import org.apache.wicket.model.StringResourceModel;
    4443
    4544/**

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/java/eu/clarin/cmdi/vlo/wicket/panels/search/SimpleSearchBrowsePanel.java

    r5032 r6267  
    2424import eu.clarin.cmdi.vlo.service.solr.SolrDocumentService;
    2525import eu.clarin.cmdi.vlo.wicket.model.FacetFieldsModel;
     26import eu.clarin.cmdi.vlo.wicket.model.SolrFieldDescriptionModel;
    2627import eu.clarin.cmdi.vlo.wicket.model.SolrFieldNameModel;
    2728import eu.clarin.cmdi.vlo.wicket.pages.FacetedSearchPage;
    … …  
    3031import org.apache.wicket.ajax.AjaxRequestTarget;
    3132import org.apache.wicket.ajax.markup.html.AjaxFallbackLink;
     33import org.apache.wicket.behavior.AttributeAppender;
    3234import org.apache.wicket.markup.html.WebMarkupContainer;
    3335import org.apache.wicket.markup.html.basic.Label;
    … …  
    164166                }
    165167            };
    166             select.add(new Label("name",
    167                     // friendly facet name based on name in FacetField
    168                     new SolrFieldNameModel(new PropertyModel(item.getModel(), "name"))));
     168           
     169            // add name label (with a description title attribute)
     170            final PropertyModel facetNameModel = new PropertyModel(item.getModel(), "name");
     171            // wrap in field name model to get a friendly facet name based on name in FacetField
     172            final Label name = new Label("name", new SolrFieldNameModel(facetNameModel));
     173            select.add(name);
     174           
     175            // add title attribute to get the facet description in a tooltip
     176            select.add(new AttributeAppender("title", new SolrFieldDescriptionModel(facetNameModel)));
     177           
    169178            item.add(select);
    170179

  • vlo/branches/vlo-ticket761/vlo-web-app/src/main/less/eu/clarin/cmdi/vlo/wicket/pages/inc/vlo-search.less

    r5354 r6267  
    6060}
    6161
    62 #searchresultsheader .searchresultpagination {
     62#searchresultsheader .resultpagination {
    6363    padding: 0;
    6464    margin: 0;
    … …  
    6969}
    7070
    71 #bottompagination {
     71.resultpagination {
     72
     73    span
     74    {
     75        padding-left: 1px;
     76        padding-right: 1px
     77    }
     78
     79    span.goto em {
     80        font-weight: bold;
     81        text-decoration: underline;
     82        font-style: normal;
     83    }
     84}
     85
     86#bottompagination, #resourcespagination {
    7287    clear: both;
    7388    text-align: center;
    7489    padding: 1em 0;
    75 }
    76 
    77 .searchresultpagination span
    78 {
    79     padding-left: 1px;
    80     padding-right: 1px
    81 }
    82 
    83 .searchresultpagination span.goto em {
    84     font-weight: bold;
    85     text-decoration: underline;
    86     font-style: normal;
    8790}
    8891
    … …  
    105108    border-bottom: 1px solid #999;
    106109    margin: 0px;
    107 }
    108 
    109 li.searchresultitem a {
    110     color: #000;
    111 }
    112 
    113 li.searchresultitem h3 {
    114     text-decoration: none;
    115     float: left;
    116     width: 75%;
    117     font-weight: normal;
    118 }
    119 
    120 li.searchresultitem h3 a {
    121     color: #000A83;
    122 }
    123 
    124 li.searchresultitem .searchresultcollection{
    125     width: 20%;
    126     float: right;
    127     text-align: right;
    128     font-weight: bold;
     110
     111    a {
     112        color: #000;
     113    }
     114
     115    h3 {
     116        text-decoration: none;
     117        float: left;
     118        width: 75%;
     119        font-weight: normal;
     120    }
     121
     122    h3 a {
     123        color: #000A83;
     124    }
     125
     126    .searchresultcollection{
     127        width: 20%;
     128        float: right;
     129        text-align: right;
     130        font-weight: bold;
     131    }
     132   
     133    .score {
     134        color: #999;
     135        text-align: right;
     136    }
    129137}
    130138

  • vlo/branches/vlo-ticket761/vlo-web-app/src/test/java/eu/clarin/cmdi/vlo/service/impl/UriResolverImplTest.java

    r4983 r6267  
    1717package eu.clarin.cmdi.vlo.service.impl;
    1818
    19 import eu.clarin.cmdi.vlo.service.handle.HandleClient;
     19import java.net.URI;
     20import nl.mpi.archiving.corpusstructure.core.handle.HandleResolver;
     21import nl.mpi.archiving.corpusstructure.core.handle.InvalidHandleException;
    2022import org.jmock.Expectations;
    2123import static org.jmock.Expectations.returnValue;
    … …  
    3436    private final Mockery context = new JUnit4Mockery();
    3537    private UriResolverImpl instance;
    36     private HandleClient handleClient;
     38    private HandleResolver handleClient;
    3739
    3840    @Before
    3941    public void setUp() {
    40         handleClient = context.mock(HandleClient.class);
     42        handleClient = context.mock(HandleResolver.class);
    4143        instance = new UriResolverImpl(handleClient);
    4244    }
    4345
    4446    /**
    45      * Test of resolve method, of class UriResolverImpl.
     47     * Test of resolve method, of class HandleClientUriResolverImpl.
    4648     */
    4749    @Test
    … …  
    5254
    5355    /**
    54      * Test of resolve method, of class UriResolverImpl.
     56     * Test of resolve method, of class HandleClientUriResolverImpl.
    5557     */
    5658    @Test
    57     public void testResolveHandleScheme() {
     59    public void testResolveHandleScheme() throws InvalidHandleException {
    5860        context.checking(new Expectations() {
    5961            {
    60                 oneOf(handleClient).getUrl("1234/5678");
    61                 will(returnValue("http://www.clarin.eu"));
     62                oneOf(handleClient).resolve(URI.create("hdl:1234/5678"));
     63                will(returnValue(URI.create("http://www.clarin.eu")));
    6264            }
    6365        });
    … …  
    6769
    6870    /**
    69      * Test of resolve method, of class UriResolverImpl.
     71     * Test of resolve method, of class HandleClientUriResolverImpl.
    7072     */
    7173    @Test
    72     public void testResolveHandleProxy() {
     74    public void testResolveHandleProxy() throws InvalidHandleException {
    7375        context.checking(new Expectations() {
    7476            {
    75                 oneOf(handleClient).getUrl("1234/5678");
    76                 will(returnValue("http://www.clarin.eu"));
     77                oneOf(handleClient).resolve(URI.create("hdl:1234/5678"));
     78                will(returnValue(URI.create("http://www.clarin.eu")));
    7779            }
    7880        });

  • vlo/branches/vlo-ticket761/vlo-web-app/src/test/java/eu/clarin/cmdi/vlo/wicket/pages/TestRecordPage.java

    r5231 r6267  
    5151    public void setUp() {
    5252        tester = new WicketTester(application);
     53       
    5354        document = new SolrDocument();
     55        document.setField(FacetConstants.FIELD_ID, "documentId");
     56       
    5457        params = new PageParameters();
    5558        params.set(VloWebAppParameters.DOCUMENT_ID, "documentId");
Note: See TracChangeset for help on using the changeset viewer.