Changes between Version 4 and Version 5 of OAIHarvester


Ignore:
Timestamp:
10/12/16 07:42:20 (8 years ago)
Author:
Menzo Windhouwer
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OAIHarvester

    v4 v5  
    5555
    5656* [[mwindhouwer|Menzo Windhouwer]]
     57* Tomasz Naskret
    5758
    5859----
     
    130131
    131132Planning and roadmap:
    132 * switch to the !ListRecords scenario, where batches of records are requested from the providers
    133 * get rid of OCLC harvester2 library, which prevents specific timeouts etc. per endpoint
    134 * get rid of always building a DOM, which blows up memory consumption
    135133* create a new OAI harvester viewer
    136134
     
    144142* can also provide access to the archived harvests
    145143
    146 Additionaly the viewer can also be an access point for tools to assess the quality of the CMD records:
    147 * run XSD validation on the records/a record
    148 * run Schematron rules agains the records/a record, e.g., to check against best practices
    149 * run the VLO importer to see if the records/a record would be included in the VLO and which facet values it would deliver
    150 * check the profiles used, e.g., in CMDI 1.2 one could check if deprecated profiles are used or now already how well they cover the VLO facets
    151 * calculate a quality score (see [http://www.lrec-conf.org/proceedings/lrec2014/pdf/1011_Paper.pdf LREC 2014 paper])
    152 
    153 These tools could by run by default on all records or allow to select a specific record to check, but also allow the upload of a record.
     144Additionaly the viewer can also be an access point for tools to assess the quality of the CMD records, i.e., tools from the Metadata Curation TF.
    154145
    155146More in the OAI domain we could also trigger a run against a OAI validator, e.g, [http://validator.oaipmh.com/], and/or allow to trigger a harvest for a specific endpoint. The latter might need a specific setup/installation to not interfere with the periodic CLARIN harvest.