source: SMC4LRT/chapters/Design_SMCinstance.tex @ 3776

\chapter{Mapping on instance level,\\ CMD as LOD}
\label{ch:design-instance}

\begin{quotation}
I do think that ISOcat, CLAVAS, RELcat and actual language
resource all provide a part of the semantic network. 

And if you can express these all in RDF, which we can for almost all of them (maybe
except the actual language resource ... unless it has a schema adorned
with ISOcat DC references ... \textless insert a SCHEMAcat plug ;-) \textgreater, but for
metadata we have that in the CMDI profiles ...) you could load all the
relevant parts in a triple store and do your SPARQL/reasoning on it. Well
that's where I'm ultimately heading with all these registries related to
semantic interoperability ... I hope ;-)

\hfill \textit{Menzo Windhouwer} \cite{Menzo2013mail}
\end{quotation}


As described in previous chapters (\ref{ch:infra},\ref{ch:design}), semantic interoperability is one of the main motivations for the CMD infrastructure. However, the established machinery pertains mostly to the schema level, the actual values in the fields of CMD instances remain ``just strings''. This is the case even though the problem of different labels for semantically equivalent or even identical entities is even more so virulent on the instance level. While for a number of metadata fields the value domain can be enforced through schema validation, some important fields (like \concept{organization} or \concept{resource type})  have a constrained value domain that yet cannot be explicitly exhaustively enumerated. This leads to a chronically inconsistent use of labels for referring to entities (as the instance data shows, some organizations are referred to by more than 20 different labels, or spelling variants.) prompting an urgent need for better means for harmonizing the constrained-field values. 

One potential remedy is the use of reference datasets -- controlled vocabularies, taxonomies, ontologies and such. In fact, this is a very common approach, be it the authority files in libraries world, or domain-specific reference vocabularies maintained by practically every research community. Not as strict as schema definitions, they cannot be used for validation, but still help to harmonize the data, by offering preferred labels and identifiers for entities.

In this chapter, we explore how this general approach can be employed for our specific problem of harmonizing the (literal) values in selected instance fields and mapping them to entities defined in corresponding vocabularies. This proposal is furthermore embedded in a more general effort to \textbf{express the whole of the CMD data domain (model and instances) in RDF} constituting one large ontology interlinked with existing external semantic resources (ontologies, knowledge bases, vocabularies). This result lays a foundation for providing the original dataset as a \emph{Linked Open Data} nucleus within the \emph{Web of Data}\cite{TimBL2006} 
as well as for real semantic (ontology-driven) search and exploration of the data.

The following section \ref{sec:cmd2rdf} lays out how individual parts of the CMD framework can be expressed in RDF.
In \ref{sec:values2entities} we investigate in further detail the abovementioned critical aspect of the effort, namely the task of translating the string values in metadata fields to corresponding semantic entities. Finally, the technical aspects of providing the resulting ontology as LOD and the implications for an ontology-driven semantic search are tackled briefly in \ref{sec:lod} and \ref{semantic-search} respectively.

\section{CMD to RDF}
\label{sec:cmd2rdf}
In this section, RDF encoding is proposed for all levels of the CMD data domain: 

\begin{itemize}
\item CMD meta model
\item profile definitions
\item the administrative and structural information of CMD records
\item individual values in the fields of the CMD records
\end{itemize}

\subsection{CMD specification}

The main entity of the meta model is the CMD component and is typed as specialization of the \code{rdfs:Class}. CMD profile is basically a CMD component with some extra features, implying a specialization relation. It would be natural to translate a CMD element to a RDF property, but it needs to be a class as a CMD element -- next to its value -- can also have attributes. This further implies a property ElementValue to express the actual value of given CMD element.

\label{table:rdf-spec}
\begin{example3}
cmds:Component & a  & rdfs:Class. \\
cmds:Profile & rdfs:subClassOf  & cmds:Component. \\
cmds:Element & a  & rdfs:Class. \\
cmds:ElementValue & a & rdf:Property \\
cmds:Attribute & a & rdf:Property \\
\end{example3}


\noindent
This entities are used for typing the actual profiles, components and elements (as they are defined in the Component Registry):

\label{table:rdf-cmd}
\begin{example3}
cmd:collection & a & cmds:Profile; \\
 & rdfs:label & "collection"; \\
 & dcterms:identifier & cr:clarin.eu:cr1:p\_1345561703620. \\
cmd:Actor       & a & cmds:Component. \\
cmd:Actor.LanguageName  & a & cmds:Element. \\
\end{example3}

%\begin{note}
%Should the ID assigned in the Component Registry  for the CMD entities be used as identifier in RDF, or rather the verbose name? (if yes, how to ensure uniqueness – generate the name from the cmd-path?)
%\end{note}


\subsection{Data Categories}
Windhouwer \cite{Windhouwer2012_LDL} proposes to use the data categories as annotation properties:

\begin{example3}
dcr:datcat & a  & owl:AnnotationProperty ; \\
 & rdfs:label  & "data category"@en ; \\
 & rdfs:comment  & "This resource is equivalent to this data category."@en ; \\
 & skos:note  & "The data category should be identified by its PID."@en ; \\
\end{example3}

That implies that the \code{@ConceptLink} attribute on CMD elements and components as used in the CMD profiles to reference the data category would be modelled as: 

\begin{example3}
cmd:LanguageName & dcr:datcat & isocat:DC-2484. \\
\end{example3}

Encoding data categories as annotation properties is in contrast to the common approach seen with dublincore terms
used usually directly as data properties:

\begin{example3}
<lr1> & dc:title & "Language Resource 1"
\end{example3}

\noindent
However, we argue against direct mapping of complex data categories to data properties and in favour of modelling data categories as annotation properties, so as to avoid too strong semantic implications.\cite{Windhouwer2012_LDL} 
In a specific (OWL 2) application the relation with the data categories can be expressed as \code{owl:equivalentClass} for classes, \code{owl:equivalentProperty} for properties or \code{owl:sameAs} for individuals:

\begin{example3} 
\#myPOS & owl:equivalentClass & isocat:DC-1345. \\
\#myPOS & owl:equivalentProperty & isocat:DC-1345. \\
\#myNoun & owl:sameAs & isocat:DC-1333. \\
\end{example3} 


\subsection{RELcat - Ontological relations}
As described in \ref{def:rr} relations between data categories are not stored directly in the \xne{ISOcat} DCR, but rather in a dedicated module the Relation Registry \xne{RELcat}. The relations here are grouped into relation sets and stored as RDF triples\cite{SchuurmanWindhouwer2011}. A sample relation from the \xne{CMDI} relation set expressing a number of equivalences between \xne{ISOcat} data categories and \xne{dublincore} terms: 

\begin{example3}
isocat:DC-2538 & rel:sameAs & dct:date
\end{example3}

\noindent
By design, the relations in Relation Registry are not expressed with predicates from known vocabularies like \xne{SKOS} or \xne{OWL}, again with the aim to avoid too strong semantic implications. This leaves leeway for further specialization of the relations in specific applications. The \code{rel:*} properties can be undrestood as an upper layer of a taxonony of relation types, implying a subtyping:

\begin{example3}
rel:sameAs & rdfs:subPropertyOf & owl:sameAs
\end{example3}



\subsection{CMD instances}
In the next step, we want to express the individual CMD instances, the metadata records, making use of the previously defined entities on the schema level, but also entities from external ontologies.

\subsubsection {Resource Identifier}

It seems natural to use the PID of a Language Resource ( \code{<lr1>} ) as the resource identifier for the subject in the RDF representation. While this seems semantically sound, not every resource has to have a PID. (This is especially the case for ``virtual'' resources like collections, that are solely defined by their constituents and don't have any data on their own.) As a fall-back the PID of the MD record ( \code{<lr1.cmd>}  from \code{cmd:MdSelfLink} element) could be used as the resource identifier. 
If identifiers are present for both resource and metadata, the relationship between the resource and the metadata record can be expressed as an annotation using the \xne{OpenAnnotation} vocabulary\furl{http://openannotation.org/spec/core/core.html\#Motivations}.
(Note also, that one MD record can describe multiple resources, this can be also easily accomodated in OpenAnnotation):

\begin{example3}
\_:anno1  & a & oa:Annotation; \\
 & oa:hasTarget  & <lr1a>, <lr1b>; \\
 & oa:hasBody  & <lr1.cmd>; \\
 & oa:motivatedBy  & oa:describing \\
\end{example3}

\subsubsection{Provenance}

The information from \code{cmd:Header} represents the provenance information about the modelled data:

\begin{example3}
<lr1.cmd> & dcterms:identifier  & <lr1.cmd>;  \\
 & dcterms:creator & "\var{\{cmd:MdCreator\}}";  \\
 & dcterms:publisher  & <http://clarin.eu>\\
 & dcterms:created /dcterms:modified? & "\var{\{cmd:MdCreated\}}" \\
\end{example3}

\subsubsection{Hierarchy ( Resource Proxy – IsPartOf)}
In CMD, the \code{cmd:ResourceProxyList} structure is used to express both collection hierarchy and point to resource(s) described by the MD record. This can be modelled as \xne{OAI-ORE Aggregation}\furl{http://www.openarchives.org/ore/1.0/primer\#Foundations}
:

\begin{example3}
<lr0.cmd>  & a   & ore:ResourceMap \\
<lr0.cmd> & ore:describes & <lr0.agg> \\
<lr0.agg> & a   & ore:Aggregation \\
& ore:aggregates  & <lr1.cmd>, <lr2.cmd>; \\
\end{example3}
        
\subsubsection{Components – nested structures}

For expressing the tree structure of the CMD records, i.e. the containment relation between the components a dedicated property \code{cmd:contains} is used:

\begin{example3}
\_:Actor1  & a & cmd:Actor \\
<lr1> & cmd:contains & \_:Actor1 \\
\end{example3}

\subsection{Elements, Fields, Values}
Finally, we want to integrate also the actual field values in the CMD records into the ontology.

% \subsubsection{Predicates}
As explained before, CMD elements have to be typed as \code{rdfs:Class}, the actual value expressed as \code{cmds:ElementValue} property and the corresponding data category expressed as annotation property.

Following example show the whole chains of statements from metamodel to literal value:

\begin{example3}
cmd:timeCoverage  & a   & cmds:Element \\
cmd:timeCoverageValue & a & cmds:ElementValue \\
cmd:timeCoverage  & dcr:datcat  & isocat:DC-2502 \\
<lr1> & cmd:contains & \_:timeCoverage1 \\
\_:timeCoverage1 & a & cmd:timeCoverage \\
\_:timeCoverage1 & cmd:timeCoverageValue & "19th century" \\
\end{example3}


While generating triples with literal values seems straightforward, the more challenging but also more valuable aspect is to generate object property triples with the literal values mapped to semantic entities:

\begin{example3}
\var{cmds:Element} & \var{cmds:ElementValue\_?} & \var{xsd:anyURI}\\
\_:organisation1 & cmd:OrganisationValue\_? & <org1> \\
\end{example3}

\begin{comment}
Don't we need a separate property (predicate) for the triples with object properties pointing to entities, 
i.e. \code{cmd:Organisation\_} additionally to \code{cmd:Organisation} 
\end{comment}

The mapping process is detailed in \ref{sec:values2entities}.



%%%%%%%%%%%%%%%%%
\section{Mapping field values to semantic entities}
\label{sec:values2entities}

This task is a prerequisite to be able to express also the CMD instance data in RDF. The main idea is to find entities in selected reference datasets (controlled vocabularies, ontologies) matching the literal values in the metadata records. The obtained entity identifiers are further used to generate new RDF triples. It involves following steps:

\begin{enumerate}
\item identify appropriate controlled vocabulares for individual metadata fields or data categories (manual task)
\item extract \emph{distinct data category, value pairs} from the metadata records
\item actual \textbf{lookup} of the individual literal values in given reference data (as indicated by the data category) to retrieve candidate entities, concepts 
\item assess the reliability of the match
\item generate new RDF triples with entity identifiers as object properties
\end{enumerate}

This task is basically an application of ontology mapping method.

We don't try to achieve complete ontology alignment, we just want to find 
for our ``anonymous'' concepts semantically equivalent concepts from other ontologies.
This is almost equivalent to the definition of ontology mapping function as given by \cite{EhrigSure2004, amrouch2012survey}:
``for each concept (node) in ontology A [tries to] find a corresponding concept
(node), which has the same or similar semantics, in ontology B and vice verse''.

The first two points in the above enumeration represent the steps necessary to be able to apply the ontology mapping.
The identification of appropriate vocabularies is discussed in the next subsection. In the operationalization, the identified vocabularies could be treated as one aggregated semantic resource to map all entities against. For the sake of higher precision, it may be sensible to perform the task separately for individual concepts, i.e. organisations, persons etc. and in every run consider only relevant vocabularies.

The transformation of the data has been partly described in previous section. It can be trivially automatically converted into RDF triples as :

\begin{example3}
\_:organisation1 & cmd:OrganisationValue & "MPI" \\
\end{example3}

However for the needs of the mapping task we propose to reduce and rewrite to retrieve distinct concept , value pairs (cf. figure \ref{fig:smc_cmd2lod}):

\begin{example3}
\_:1 & a & clavas:Organisation;\\
   & skos:altLabel & "MPI";
\end{example3}

\var{lookup} function is a customized version of the \var(map) function, that operates on this information pairs (concept, label).

The two steps \var{lookup} and \var{assess} correspond exactly to the two steps in \cite{jimenez2012large} in their system \xne{LogMap2}: 1) computation of mapping candidates (maximise recall) and b) assessment of the candidates (maximize precision)


\begin{figure*}[!ht]
\includegraphics[width=1\textwidth]{images/SMC_CMD2LOD}
\caption{Sketch of the process of transforming the CMD metadata records to a RDF representation}
\label{fig:smc_cmd2lod}
\end{figure*}

\subsubsection{Identify vocabularies}

One generic way to indicate vocabularies for given metadata fields or data categories being discussed in the CMD community is to use dedicated annotation property in the schema or data category definition (tentatively labeled \code{@clavas:vocabulary}) . For such a mechanism to work, the consuming applications (like metadata editor) need to be made aware of this convention and interpret it accordingly.

The primary provider of relevant vocabularies is \xne{ISOcat} and \xne{CLAVAS} – a service for managing and providing vocabularies in SKOS format (cf. \ref{def:CLAVAS}). Closed and corresponding simple data categories are already being exported from ISOcat in SKOS format and imported into CLAVAS/OpenSKOS and also other relevant vocabularies shall be ingested into this system, so that we can assume OpenSKOS as a first source of vocabularies. However definitely not all of the existing reference data will be hosted by OpenSKOS, so in general we have to assume/consider a number of different sources (cf. \ref{refdata}).

Data in OpenSKOS is modelled purely in SKOS, so there is no more specific typing of the entities in the vocabularies, but rather all the entities are \code{skos:Concepts}:

\begin{example3}
<org1> & a   & skos:Concept \\
\end{example3}

\noindent
We may want to add some more typing and introduce classes for entities from individual vocabularies like \code{clavas:Organization} or similar. As far as CLAVAS will also maintain mappings/links to other datasets

\begin{example3}
<org1> & skos:exactMatch  & <dbpedia/org1>, <lt-world/orgx>;
\end{example3}

\noindent
we could use it to expand the data with alternative identifiers, fostering the interlinking of data:

\begin{example3}
<org1>  & dcterms:identifier  & <org1>, <dbpedia/org1>, <lt-world/orgx>;
\end{example3}

\subsubsection{Lookup}

In abstract term, the lookup function takes as input the identifier of data category (or CMD element) and a literal string value and returns a list of potentially matching entities. Before actual lookup, there may have to be some string-normalizing preprocessing.

\begin{definition}{signature of the lookup function}
lookup \ ( \ DataCategory \ ,  \ Literal \ )  \quad \mapsto \quad ( \ Concept \ | \ Entity \ )*
\end{definition}

In the implementation there needs to be additional initial configuration input, identifying datasets for given data categories, 
which will be the result of the previous step.

\begin{definition}{Required configuration data indicating data category to available }
DataCategory \quad \mapsto \quad Dataset+
\end{definition}

As for the implementation, in the initial setup the system could resort to the \code{find}-interface provided by \xne{OpenSKOS}.
However, in the long term a more general solution is required, a kind of hybrid \emph{vocabulary proxy service} that allows to search in a number of datasets, many of them distributed and available via different interfaces. Figure \ref{fig:vocabulary_proxy} sketches the general setup. The service has to be able to a) proxy search requests to a number of search interfaces (SRU, SPARQL), b) fetch, cache and search in datasets.

\begin{figure*}[!ht]
\includegraphics[width=1\textwidth]{images/VocabularyProxy_clientapp}
\caption{Sketch of a general setup for vocabulary lookup via a \xne{VocabularyProxy} service}
\label{fig:vocabulary_proxy}
\end{figure*}

\subsubsection{Candidate evaluation}
The lookup is the most sensitive step in the process, as that is the gate between strings and semantic entities. In general, the resulting candidates cannot be seen as reliable and should undergo further scrutiny to ensure that the match is semantically correct.

One example: A lookup with the pair \code{<organization, "Academy of sciences">} would probably return a list of organizations, as there is a national Academy of Sciences, in a number of countries. It would require further heuristics, e.g. checking the corresponding department, contact or -- less reliably -- the language of the described resource, to determine which specific Academy of Sciences is meant in given resource description.

In some situation this ambiguities can be resolved algorithmically, but in the end in many cases it will require human curation of the generated data. In this respect, it is worth to note, that the CLARIN search engine VLO provides a feedback link, that allows even the normal user to report on problems or inconsistencies in CMD records.



%%%%%%%%%%%%%%%%%%%%%
\section{SMC LOD - Semantic Web Application}
\label{sec:lod}

With the new enhanced dataset, as detailed in section \ref{sec:cmd2rdf}, the groundwork is laid for the full-blown semantic search as proposed in the original goals, i.e. the possibility of exploring the dataset using external semantic resources.
The user can access the data indirectly by browsing external vocabularies/taxonomies, with which the data will be linked like vocabularies of organizations or taxonomies of resource types.

The technical base for a semantic web application is usually a RDF triple-store as discussed in \ref{semweb-tech}.
Given that our main concern is the data itself, their processing and display, we want to rely on stable, robust feature rich solution minimizing the effort to provide the data online. The most promising solution seems to be \xne{Virtuoso}, a integrated feature-rich hybrid data store, able to deal with different types of data (``Universal Data Store'').


Although the distributed nature of the data is one of the defining features of LOD and theoretically one should be able to follow the data by dereferencable URIs, in practice it is mostly necessary to pool into one data store linked datasets from different sources that shall be queried together due to performance reasons. This implies that the data to be kept by the data store will be decisively larger, than ``just'' the original dataset.

\section{Summary}

In this chapter, an expression of the whole of the CMD data domain into RDF was proposed, with special focus on the method to translate the string values in metadata fields to corresponding semantic entities. 
This task can be also seen as building a bridge between the world XML resources and semantic resources expressed in RDF.
Additionally, some technical considerations were discussed regarding exposing this dataset as Linked Open Data and the implications for real semantic ontology-based data exploration.

%The process of expressing the whole of the data as one semantic resource, can be also understood as schema or ontology merging task. Data categories being the primary mapping elements