Changes between Version 3 and Version 4 of CMD2RDF/sysarch

Timestamp:

05/21/14 14:31:06 (10 years ago)

Author:

Menzo Windhouwer

Comment:

--

CMD2RDF/sysarch

@@ -75 +75 @@
 1. Process the CMD records (in multiple threads)
   a. for each record process the actions sequentially, where the output of each action is the input of the next
-  b. the result of the last action should be RDF XML to be uploaded to the Graph Store
+  b. the input of the first action should be a CMD record from the OAI harvester result set
+  c. the result of the last action should be RDF XML to be uploaded to the Graph Store
 2. Process the profiles (as used by the records) (in multiple threads)
   a. for each profile process the actions sequentially
-  b. the result of the last action should be RDF XML to be uploaded to the Graph Store
+  b. the input of the first action is a CMD profile from the CMD2RDF cache
+  c. the result of the last action should be RDF XML to be uploaded to the Graph Store
 3. Process the components (as used by the profiles) (in multiple threads)
   a. for each component process the actions sequentially
-  b. the result of the last action should be RDF XML to be uploaded to the Graph Store
+  b. the input of the first action is a CMD component from the CMD2RDF cache
+  c. the result of the last action should be RDF XML to be uploaded to the Graph Store
 
 '''Note''': for phase 3 phase 2 could output the components, which might complicated as until now they would just request resources from the cache not store them. Phase 2 could also just request the used components from the Component Registry and thus trigger caching from them. This is some extra load for the Component Registry, but might keep the CMD2RDF caching API simpler.
@@ -89 +92 @@
 For inspiration: the [[https://github.com/TheLanguageArchive/oai-harvest-manager|OAI harvester]] allows to configure actions on harvested records, e.g., to convert OLAC to CMDI.
 
+'''TODO:''' the harvest isn't incremental, but CMD2RDF could be. It could handle only the delta of a previous run, i.e., convert and enrich new CMD records, convert and enrich updated CMD records, remove graphs for deleted CMD records, the same for profiles and components. MD5 checksums might help to determine if CMD records have been changed ...
+
+
 = CMD-RDF