source: SMC4LRT/chapters/Design_SMCinstance.tex @ 3671

\chapter{Mapping on instance level,\\ CMD as LOD}
\label{ch:design-instance}

\begin{quotation}
I do think that ISOcat, CLAVAS, RELcat and actual language
resource all provide a part of the semantic network. 

And if you can express these all in RDF, which we can for almost all of them (maybe
except the actual language resource ... unless it has a schema adorned
with ISOcat DC references ... \textless insert a SCHEMAcat plug ;-) \textgreater, but for
metadata we have that in the CMDI profiles ...) you could load all the
relevant parts in a triple store and do your SPARQL/reasoning on it. Well
that's where I'm ultimately heading with all these registries related to
semantic interoperability ... I hope ;-)\cite{Menzo2013mail}
\end{quotation}

As described in previous chapters (\ref{ch:infra},\ref{ch:design}), semantic interoperability is one of the main motivations for the CMD infrastructure. However, the established machinery pertains mostly to the schema level, the actual values in the fields of CMD instances remain ``just strings''. This is the case even though the problem of different labels for semantically equivalent or even identical entities is even more so virulent on the instance level. While for a number of metadata fields the value domain can be enforced through schema validation, some important fields (like \concept{organization} or \concept{resource type})  have a constrained value domain that yet cannot be explicitly exhaustively enumerated. This leads to a chronically inconsistent use of labels for referring to entities (as the instance data shows, some organizations are referred to by more than 20 different labels, or spelling variants.) prompting an urgent need for better means for harmonizing the constrained-field values. 

One potential remedy is the use of reference datasets -- controlled vocabularies, taxonomies, ontologies and such. In fact, this is a very common approach, be it the authority files in libraries world, or domain-specific reference vocabularies maintained by practically every research community. Not as strict as schema definitions, they cannot be used for validation, but still help to harmonize the data, by offering preferred labels and identifiers for entities.

In this chapter, we explore how this general approach can be employed for our specific problem of harmonizing the (literal) values in selected instance fields and mapping them to entities defined in corresponding vocabularies. This proposal is furthermore embedded in a more general effort to \textbf{express the whole of the CMD data domain (model and instances) in RDF} constituting one large ontology interlinked with existing external semantic resources (ontologies, knowledge bases, vocabularies). This result lays a foundation for providing the original dataset as a \emph{Linked Open Data} nucleus within the \emph{Web of Data}\cite{TimBL2006} 
as well as for real semantic (ontology-driven) search and exploration of the data.

The following section \ref{sec:cmd2rdf} lays out how individual parts of the CMD framework can be expressed in RDF.
In \ref{sec:values2entities} we investigate in further detail the abovementioned critical aspect of the effort, namely the task of translating the string values in metadata fields to corresponding semantic entities. Finally, the technical aspects of providing the resulting ontology as LOD and the implications for an ontology-driven semantic search are tackled briefly in \ref{sec:lod} and \ref{semantic-search} respectively.

\section{CMD to RDF}
\label{sec:cmd2rdf}
In this section, RDF encoding is proposed for all levels of the CMD data domain: 

\begin{itemize}
\item CMD meta model
\item profile definitions
\item the administrative and structural information of CMD records
\item individual values in the fields of the CMD records
\end{itemize}

\subsection{CMD specification}

The main entity of the meta model is the CMD component and is typed as specialization of the \code{owl:Class}. CMD profile is basically a CMD component with some extra features, implying a specialization relation:

\label{table:rdf-spec}
\begin{example3}
cmds:Component & subClassOf  & owl:Class. \\
cmds:Profile & subClassOf  & cmds:Component. \\
cmds:Element & subClassOf  & rdf:Property. \\
\end{example3}

\noindent
This entities are used for typing the actual profiles, components and elements (as they are defined in the Component Registry):

\label{table:rdf-cmd}
\begin{example3}
cmd:collection & a & cmds:Profile; \\
 & rdfs:label & "collection"; \\
 & dcterms:identifier & cr:clarin.eu:cr1:p\_1345561703620. \\
cmd:Actor       & a & cmds:Component. \\
cmd:LanguageName  & a & cmds:Element. \\
\end{example3}

\begin{note}
Should the ID assigned in the Component Registry  for the CMD entities be used as identifier in RDF, or rather the verbose name? (if yes, how to ensure uniqueness – generate the name from the cmd-path?)
\end{note}

\subsection{Data Categories}
Windhouwer \cite{Windhouwer2012_LDL} proposes to use the data categories as annotation properties:

\begin{example3}
dcr:datcat & a  & owl:AnnotationProperty ; \\
 & rdfs:label  & "data category"@en ; \\
 & rdfs:comment  & "This resource is equivalent to  this data category."@en ; \\
 & skos:note  & "The data category should be identified by its PID."@en ; \\
\end{example3}

That implies that the \code{@ConceptLink} attribute on CMD elements and components as used in the CMD profiles to reference the data category would be modelled as: 

\begin{example3}
cmd:LanguageName & dcr:datcat & isocat:DC-2484. \\
\end{example3}

Encoding data categories as annotation properties is in contrast to the common approach seen with dublincore terms
used usually directly as data properties:

\begin{example3}
<lr1> & dc:title & "Language Resource 1"
\end{example3}

\noindent
Analogously, we could model \xne{ISOcat} data categories as data properties, i.e. metadata elements referencing ISOcat data categories could be encoded as follows:

\begin{example3}
<lr1> & isocat:DC-2502 & "19th century"
\end{example3}

\noindent
However, Windhouwer\cite{Windhouwer2012_LDL} argues against direct mapping of complex data categories to data properties and in favour of modelling data categories as annotation properties, so as to avoid too strong semantic implications.

This raises the vice-versa question, whether to rather handle all data categories uniformly, which would mean encoding dublincore terms also as annotation properties, but the pragmatic view dictates to encode the data in line with the prevailing approach, i.e. express dublincore terms directly as data properties.


\noindent
The REST web service of \xne{ISOcat} provides a RDF representation of the data categories:

\begin{example3}
isocat:languageName & dcr:datcat & isocat:DC-2484; \\
 & rdfs:label & "language name"@en; \\
 & rdfs:comment & "A human understandable..."@en; \\
 & 
  \\
\end{example3}

However this is only meant as template, as is stated in the explanatory comment of the exported data:

\begin{quotation}
By default the RDF export inserts \code{dcr:datcat} annotation properties to maintain the link between the generated RDF resources and the used Data Categories. However, it is possible to also maintain a stronger semantic link when the RDF resources will be used as OWL (2) classes, properties or individuals.
\end{quotation}

So in a specific (OWL 2) application the relation with the data categories can be expressed as \code{owl:equivalentClass} for classes, \code{owl:equivalentProperty} for properties or \code{owl:sameAs} for individuals:

\begin{example3} 
\#myPOS & owl:equivalentClass & isocat:DC-1345. \\
\#myPOS & owl:equivalentProperty & isocat:DC-1345. \\
\#myNoun & owl:sameAs & isocat:DC-1333. \\
\end{example3} 


\subsection{RELcat - Ontological relations}
As described in \ref{def:rr} relations between data categories are not stored directly in the \xne{ISOcat} DCR, but rather in a dedicated module the Relation Registry \xne{RELcat}. The relations here are grouped into relation sets and stored as RDF triples\cite{SchuurmanWindhouwer2011}. A sample relation from the \xne{CMDI} relation set expressing a number of equivalences between \xne{ISOcat} data categories and \xne{dublincore} terms: 

\begin{example3}
isocat:DC-2538 & rel:sameAs & dct:date
\end{example3}

\noindent
By design, the relations in Relation Registry are not expressed with predicates from known vocabularies like \xne{SKOS} or \xne{OWL}, again with the aim to avoid too strong semantic implications. This leaves leeway for further specialization of the relations in specific applications.

\begin{note}
Does this mean, that I would say:
\begin{example3}
rel:sameAs & owl:equivalentProperty & owl:sameAs
\end{example3}

to enable the inference of the equivalences?

Is this correct:
\end{note}
?? That means, that to be able to infer that a value in a CMD element also pertains to a given data category, e.g.:

\begin{example2}
 cmd:PublicationYear = 2012 $\rightarrow$ & dc:created = 2012 
\end{example2}

\noindent
following facts need to be present in the ontology :

\begin{example3}
<lr1> & cmd:PublicationYear & 2012\^{}\^{}xs:year \\
cmd:PublicationYear &  owl:equivalentProperty & isocat:DC-2538 \\
isocat:DC-2538 & rel:sameAs & dc:created \\
rel:sameAs & owl:equivalentProperty &  owl:sameAs \\
$\rightarrow$ \\
<lr1> & dc:created & 2012\^{}\^{}xs:year \\
\end{example3}

\noindent
What about other relations we may want to express? (Do we need them and if yes, where to put them? – still in RR?) Examples:

\begin{example3}
cmd:MDCreator   & owl:subClassOf & dcterms:Agent \\
clavas:Organization & owl:subClassOf & dcterms:Agent \\
<org1> & a & clavas:Organization \\
\end{example3}

\subsection{CMD instances}
In the next step, we want to express the individual CMD instances, the metadata records, making use of the previously defined entities on the schema level, but also entities from external ontologies.

\subsubsection {Resource Identifier}

It seems natural to use the PID of a Language Resource ( \code{<lr1>} ) as the resource identifier for the subject in the RDF representation. While this seems semantically sound, not every resource has to have a PID. (This is especially the case for ``virtual'' resources like collections, that are solely defined by their constituents and don't have any data on their own.) As a fall-back the PID of the MD record ( \code{<lr1.cmd>}  from \code{cmd:MdSelfLink} element) could be used as the resource identifier. 
If identifiers are present for both resource and metadata, the relationship between the resource and the metadata record can be expressed as an annotation using the \xne{OpenAnnotation} vocabulary\furl{http://openannotation.org/spec/core/core.html\#Motivations}:

\begin{example3}
\_:anno1  & a & oa:Annotation; \\
 & oa:hasTarget  & <lr1>; \\
 & oa:hasBody  & <lr1.cmd>; \\
 & oa:motivatedBy  & oa:describing \\
\end{example3}

\subsubsection{Provenance}

The information from \code{cmd:Header} represents the provenance information about the modelled data:

\begin{example3}
<lr1.cmd> & dcterms:identifier  & <lr1.cmd>;  \\
 & dcterms:creator ??  & "\var{\{cmd:MdCreator\}}";  \\
 & dcterms:publisher  & <http://clarin.eu>, <provider-oai-accesspoint>; ?? \\
 & dcterms:created /dcterms:modified? & "\var{\{cmd:MdCreated\}}" ?? \\
\end{example3}

\subsubsection{Hierarchy ( Resource Proxy – IsPartOf)}
In CMD, the \code{cmd:ResourceProxyList} structure is used to express both collection hierarchy and point to resource(s) described by the MD record. This can be modelled as \xne{OAI-ORE Aggregation}\furl{http://www.openarchives.org/ore/1.0/primer\#Foundations}
:

\begin{example3}
<lr0.cmd>  & a   & ore:ResourceMap \\
<lr0.cmd> & ore:describes & <lr0.agg> \\
<lr0.agg> & a   & ore:Aggregation \\
& ore:aggregates  & <lr1.cmd>, <lr2.cmd>; \\
\end{example3}

\noindent
?? Should both collection hierarchy and resource-pointers (collection and resource MD records) be encoded as ore:Aggregation?
Additionally the flat header field \code{cmd:MdCollectionDisplayName} has been introduced to indicate by simple means the collection, of which given resource is part.
This information can be used to generate a separate one-level grouping of the resources, in which the value from the \code{cmd:MdCollectionDisplayName} element would be used as the label of an otherwise undefined \code{ore:ResourceMap}.
Even the identifier/ URI for this collections is not clear. Although this collections should match with the ResourceProxy hierarchy, there is no guarantee for this, thus a 1:1 mapping cannot be expected.

\todocode{check consistency for MdCollectionDisplayName vs. IsPartOf in the instance data}

\begin{example3}
\_:mdcoll  & a   & ore:ResourceMap; \\
 & rdfs:label & "Collection 1"; \\
\_:mdcoll\#aggreg & a   & ore:Aggregation \\
 & ore:aggregates  & <lr1.cmd>, <lr2.cmd>; \\
\end{example3}
        
\subsubsection{Components – nested structures}

There are two variants to express the tree structure of the CMD records, i.e. the containment relation between the components:

\begin{enumerate}[a)]
\item the components are encoded as object property

\begin{example3}
<lr1>  & cmd:Actor  & \_:Actor1 \\
<lr1>  & cmd:Actor  & \_:Actor2 \\
\_:Actor1  & cmd:motherTongue  & iso-639:aac \\
\_:Actor2  & cmd:motherTongue  & iso-639:deu \\
\_:Actor1  & cmd:role & "Interviewer" \\
\_:Actor2 & cmd:role & "Speaker" \\
\end{example3}

\item a dedicated object property is used

\begin{example3}
\_:Actor1  & a & cmd:Actor \\
<lr1> & cmd:contains & \_:Actor1 \\
\end{example3}

\end{enumerate}

\subsection{Elements, Fields, Values}
Finally, we want to integrate also the actual field values in the CMD records into the ontology.

\subsubsection{Predicates}
As explained before CMD elements are typed as \code{rdf:Property} with the corresponding data category expressed as annotation property:

\begin{example3}
cmd:timeCoverage  & a   & cmds:Element \\
cmd:timeCoverage  & dcr:datcat  & isocat:DC-2502 \\
<lr1>  & cmd:timeCoverage  & "19th century" \\

\end{example3}

\subsubsection{Literal values -- data properties}

To generate triples with literal values is straightforward:

\begin{definition}{Literal triples}
lr:Resource \ \quad cmds:Property \ \quad xsd:string
\end{definition}

\begin{example3}
<lr1> & cmd:Organisation & "MPI" \\
\end{example3}

\subsubsection{Mapping to entities -- object properties}

The more challenging but also more valuable aspect is to generate objectProperty triples with the literal values mapped to semantic entities:

\begin{definition}{new RDF triples}
lr:Resource \ \quad cmd:Property \ \quad xsd:anyURI
\end{definition}

\begin{example3}
<lr1> & cmd:Organisation\_? & <org1> \\
\end{example3}

\begin{note}
Don't we need a separate property (predicate) for the triples with object properties pointing to entities, 
i.e. \code{cmd:Organisation\_} additionally to \code{cmd:Organisation} 
\end{note}

The mapping process is detailed in \ref{sec:values2entities}

%%%%%%%%%%%%%%%%%55
\section{Mapping field values to semantic entities}
\label{sec:values2entities}

This task is a prerequisite to be able to express also the CMD instance data in RDF. The main idea is to find entities in selected reference datasets (controlled vocabularies, ontologies) matching the literal values in the metadata records. The obtained entity identifiers are further used to generate new RDF triples. It involves following steps:

\begin{enumerate}
\item identify appropriate controlled vocabulares for individual metadata fields or data categories (manual task)
\item extract \emph{distinct data category, value pairs} from the metadata records
\item actual \textbf{lookup} of the individual literal values in given reference data (as indicated by the data category) to retrieve candidate entities, concepts 
\item assess the reliability of the match
\item generate new RDF triples with entity identifiers as object properties
\end{enumerate}

\begin{figure*}[!ht]
\includegraphics[width=1\textwidth]{images/SMC_CMD2LOD}
\caption{Sketch of the process of transforming the CMD metadata records to a RDF representation}
\label{fig:smc_cmd2lod}
\end{figure*}

\subsubsection{Identify vocabularies}

\todoin{Identify related ontologies, vocabularies? - see DARIAH:CV}
LT-World \cite{Joerg2010}

One generic way to indicate vocabularies for given metadata fields or data categories being discussed in the CMD community is to use dedicated annotation property (tentatively \code{@clavas:vocabulary}) in the schema or data category definition. For such a mechanism to work, the consuming applications (like metadata editor) need to be made aware of this convention and interpret it accordingly.

The primary provider of relevant vocabularies is \xne{ISOcat} and \xne{CLAVAS} – a service for managing and providing vocabularies in SKOS format (cf. \ref{def:CLAVAS}). Closed and corresponding simple data categories are already being exported from ISOcat in SKOS format and imported into CLAVAS/OpenSKOS and also other relevant vocabularies shall be ingested into this system, so that we can assume OpenSKOS as a first source of vocabularies. However definitely not all of the existing reference data will be hosted by OpenSKOS, so in general we have to assume/consider a number of different sources (cf. \ref{refdata}).

Data in OpenSKOS is modelled purely in SKOS, so there is no more specific typing of the entities in the vocabularies, but rather all the entities are \code{skos:Concepts}:

\begin{example3}
<org1> & a   & skos:Concept \\
\end{example3}

\noindent
We may want to add some more typing and introduce classes for entities from individual vocabularies like \code{clavas:Organization} or similar. As far as CLAVAS will also maintain mappings/links to other datasets

\begin{example3}
<org1> & skos:exactMatch  & <dbpedia/org1>, <lt-world/orgx>;
\end{example3}

\noindent
we could use it to expand the data with alternative identifiers, fostering the interlinking of data:

\begin{example3}
<org1>  & dcterms:identifier  & <org1>, <dbpedia/org1>, <lt-world/orgx>;
\end{example3}

\subsubsection{Lookup}

In abstract term, the lookup function takes as input the identifier of data category (or CMD element) and a literal string value and returns a list of potentially matching entities. Before actual lookup, there may have to be some string-normalizing preprocessing.

\begin{definition}{signature of the lookup function}
lookup \ ( \ DataCategory \ ,  \ Literal \ )  \quad \mapsto \quad ( \ Concept \ | \ Entity \ )*
\end{definition}

In the implementation, there needs to be additional initial configuration input, identifying datasets for given data categories, 
which will be the result of the previous step.

\begin{definition}{Required configuration data indicating data category to available }
DataCategory \quad \mapsto \quad Dataset+
\end{definition}

As for the implementation, in the initial setup the system could resort to the \code{find}-interface provided by \xne{OpenSKOS}.
However, in the long term a more general solution is required, a kind of hybrid \emph{vocabulary proxy service} that allows to search in a number of datasets, many of them distributed and available via different interfaces. Figure \ref{fig:vocabulary_proxy} sketches the general setup. The service has to be able to a) proxy search requests to a number of search interfaces (SRU, SPARQL), b) fetch, cache and search in datasets.

\begin{figure*}[!ht]
\includegraphics[width=1\textwidth]{images/VocabularyProxy_clientapp}
\caption{Sketch of a general setup for vocabulary lookup via a \xne{VocabularyProxy} service}
\label{fig:vocabulary_proxy}
\end{figure*}

\subsubsection{Candidate evaluation}
The lookup is the most sensitive step in the process, as that is the gate between strings and semantic entities. In general, the resulting candidates cannot be seen as reliable and should undergo further scrutiny to ensure that the match is semantically correct.

One example: A lookup with the pair \code{<organization, "Academy of sciences">} would probably return a list of organizations, as there is a national Academy of Sciences, in a number of countries. It would require further heuristics, e.g. checking the corresponding department, contact or -- less reliably -- the language of the described resource, to determine which specific Academy of Sciences is meant in given resource description.

In some situation this ambiguities can be resolved algorithmically, but in the end in many cases it will require human curation of the generated data. In this respect, it is worth to note, that the CLARIN search engine VLO provides a feedback link, that allows even the normal user to report on problems or inconsistencies in CMD records.


%%%%%%%%%%%%%%%%%%%%%
\section{SMC LOD - Semantic Web Application}
\label{sec:lod}



\cite{Europeana RDF Store Report}

Technical aspects (RDF-store?): Virtuoso

\todocode{install Jena +  fuseki}\furl{http://jena.apache.org}\furl{http://jena.apache.org/documentation/serving_data/index.html}\furl{http://csarven.ca/how-to-create-a-linked-data-site}

\todocode{install older python (2.5?) to be able to install dot2tex - transforming dot files to nicer pgf formatted graphs}\furl{http://dot2tex.googlecode.com/files/dot2tex-2.8.7.zip}\furl{file:/C:/Users/m/2kb/tex/dot2tex-2.8.7/}

\todocode{check install siren}\furl{http://siren.sindice.com/}


\todocode{check/install: raptor for generating dot out of rdf}\furl{http://librdf.org/raptor/}

\todocode{check/install: Linked Data browser: LoD p. 81; Haystack}\furl{http://en.wikipedia.org/wiki/Haystack_(PIM)}

 / interface (ontology browser?)

semantic search component in the Linked Media Framework
\todocode{!!! check install LMF - kiwi - SemanticSearch !!!}\furl{http://code.google.com/p/kiwi/wiki/SemanticSearch}

\todoin{check SARQ}\furl{http://github.com/castagna/SARQ}


\section {Full semantic search - concept-based + ontology-driven ?}
\label{semantic-search}

With the new enhanced dataset, as detailed in section \ref{sec:cmd2rdf}, the groundwork is laid for the full-blown semantic search as proposed in the original goals, i.e. the possibility for ontology-driven or at least `semantic resources assisted' exploration of the dataset.

Namely to enhance it by employing ontological resources.
Mainly this enhancement shall mean, that the user can access the data indirectly by browsing one or multiple ontologies, with which the data will then be linked. These could be for example ontologies of Organizations and Projects.


SPARQL

rechercheisidore, dbpedia, ...

\section{Summary}
In this chapter, an expression of the whole of the CMD data domain into RDF was proposed, with special focus on the way how to translate the string values in metadata fields to corresponding semantic entities. Additionally, some technical considerations were discussed regarding exposing this dataset as Linked Open Data and the implications for real semantic ontology-based data exploration.