source: SMC4LRT/SMC.tex @ 2671

\section{Semantic Mapping on concept level}

merging the pieces of information provided by those,
offering them semi-transaprently to the user (or application) on the consumption side.

a module of the Component Metadata Infrastructure performing semantic mapping on search indexes. This  builds the base for query expansion to facilitate semantic search and enhance recall when querying the Metadata Repository.


\subsection{smcIndex}\label{indexes}
In this section we describe \emph{smcIndex} -- the data type for input and output of the proposed application. 
An smcIndex is a human-readable string adhering to a specific syntax, denoting some search index. 
The generic syntax is:
\begin{eqnarray*}
smcIndex ::= context \ contextSep \ conceptLabel
\end{eqnarray*}

We distinguish two types of smcIndexes: (i) \emph{dcrIndex} referring to data categories and (ii) \emph{cmdIndex} denoting a specific
"CMD-entity", i.e. a metadata field, component or whole profile defined within CMD. The \textit{cmdIndex} can be interpreted as a XPath into the instances of CMD-profiles. In contrast to it, the \textit{dcrIndexes} are generally not directly applicable on existing data, but can be understood as abstract indexes referring to well-defined concepts -- the data categories -- and for actual search they need to be resolved to the metadata fields they are referred by. In return one can expect to match more metadata fields from multiple profiles, all referring to the same data category.

These two types of smcIndex also follow different construction patterns:
\begin{eqnarray*}
smcIndex & ::= & dcrIndex \ | \ cmdIndex  \\
dcrIndex & ::= & dcrID \ contextSep \ datcatLabel \\
cmdIndex & ::= & profile \  \\
                      &  &  | \  [\ profile \ contextSep \ ] \ dotPath \\
dotPath  & ::= & [\ dotPath \ pathSep \ ] \ elemName \\
contextSep & ::= & \texttt{`.`} \ | \  \texttt{`:`} \\
pathSep & ::= & \texttt{`.`} \\
dcrId & ::= & \texttt{`isocat`} \ | \ \texttt{`dc`}
\end{eqnarray*}

The grammar is based on the way indices are referenced in CQL-syntax\footnote{Context Query Language, \url{http://www.loc.gov/standards/sru/specs/cql.html}} (\texttt{dc.title}) and on the dot-notation used in IMDI-browser\footnote{\url{http://www.lat-mpi.eu/tools/imdi}} (\texttt{Session.Location.Country}). 

\textit{dcrID} is a shortcut referring to a data category registry
%\footnote{Next to ISOcat other registries can function as a DCR, e.g., the Dublin Core set of metadata terms.} 
similar to the namespace-mechanism in XML-documents.  \textit{datcatLabel} is the verbose Identifier- (e.g. \texttt{telephoneNumber}) or the Name-attribute (in any available translation, e.g. \texttt{numero di telefono@it}) of the data category.
% While it is desirable to also allow the Name-attribute of the data category (\texttt{telephone number}), especially also the Names defined in other working languages (\texttt{numero di telefono@it, numer telefonu@pl}), special care has to be taken here as these attributes mostly contain white spaces, which could cause problems in downstream components, when parsing a complex query containing such indices. 
\textit{profile} is the name of the profile. % (despite the danger of ambiguity). 
\textit{dotPath} allows to address a leaf element (\texttt{Session.Actor.Role}), or any intermediary XML-element corresponding to a CMD-component (\texttt{Session.Actor})   within a metadata description. %This allows to easily express search in whole components, instead of having to list all individual fields.

Generally, smcIndexes can be ambiguous, meaning they can refer to multiple concepts, or entities (CMD-elements). This is due to the fact that the names of the data categories, and CMD-entities are not guaranteed unique. The module will have to cope with this, by providing on demand the list of identifiers corresponding to a given smcIndex.

%As an important sidenote -- cmdIndexes can be ambiguous, meaning they can refer to multiple entities (metadata fields), examples of valid indexes:
%\begin{verbatim}
%Name
%Actor.Name, Project.Name
%Session.Actor.Name, Drama.Actor.Name
%\end{verbatim}

%So we disambiguate (or narrow down the ambiguity) by prefixing context.


\subsection{Function}\label{method}
In this section, we describe the actual task of the proposed application -- \textbf{mapping indexes to indexes} -- in abstract terms. The returned mappings can be used by other applications to expand or translate the original user query, to match elements in other schemas.
\footnote{Though tightly related, mapping of terms and query expansion are to be seen as two separate functions.}
% \footnote{This primary usage of SMC for work with user-created query strings explains the need for human-readability of the indices.}


\subsubsection*{Initialization}

First there is an initialization phase, in which the application fetches the information from the source modules (cf. \ref{components}). All profiles and components from the Component Registry are read and all the URIs to data categories are extracted to construct an inverted map of data categories:
\newline

\textit{datcatURI $\mapsto$ profile.component.element[]}
\newline

The collected data categories are enriched with information from corresponding registries (DCRs), adding the verbose identifier, the description and available translations into other working languages. %, usable as base for multi-lingual search user-interface.

Finally relation sets defined in the Relation Registry are fetched and matched with the data categories in the map to create sets of semantically equivalent (or otherwise related) data categories. 

\subsubsection*{Operation}
In the operation mode, the application accepts any index (\textit{smcIndex}, cf. \ref{indexes}) and returns a list of corresponding indexes (or only the input index, if no correspondences were found):
\newline

\textit{smcIndex $\mapsto$ smcIndex[ ]}
\newline

We can distinguish following levels for this mapping function:

(1) \emph{data category identity} -- for the resolution only the basic data category map derived from Component Registry is employed. Accordingly, only indexes denoting CMD-elements (\textit{cmdIndexes)} bound to a given data category are returned:
\newline

\texttt{isocat.size $\mapsto$ } \newline 
\verb|   [teiHeader.extent, |\newline 
\verb|    TextCorpusProfile.Number]|
\newline

\textit{cmdIndex} as input is also possible. It is translated to a corresponding data category, proceeding as above:
\newline

\texttt{imdi-corpus.Name   $\mapsto$ } \newline 
\verb|   (isocat.resourceName) |$\mapsto$  \newline 
\verb|   TextCorpusProfile.GeneralInfo.Name|
\newline 

(2) \emph{relations between data categories} -- employing also information from the Relation Registry, related (equivalent) data categories are retrieved and subsequently both the input and the related data categories resolved to cmdIndexes:
\newline

\texttt{isocat.resourceTitle  $\mapsto$ } 
\verb|   (+ dc.title) |$\mapsto$  \newline 
\verb|   [imdi-corpus.Title, | \newline 
\verb|    TextCorpusProfile.GeneralInfo.Title,| \newline 
\verb|    teiHeader.titleStmt.title,| \newline 
\verb|    teiHeader.monogr.title]|
\newline 

(3) \emph{container data categories} -- further expansions will be possible once the container data categories \cite{SchuurmanWindhouwer2011} will be used. Currently only fields (leaf nodes) in metadata descriptions are linked to data categories. However, at times, there is a need to conceptually bind also the components, meaning that besides the "atomic" data category for \texttt{actorName, there would be also a data category for the complex concept \texttt{Actor}.} 
Having concept links also on components will require a compositional approach to the task of semantic mapping, resulting in:
\newline 
\texttt{Actor.Name $\mapsto$ }\newline
\verb|    [Actor.Name, Actor.FullName, |\newline
\verb|     Person.Name, Person.FullName]|


\subsection*{Extensions}

A useful supplementary function of the module would be to provide a list of existing indexes.
That would allow the search user-interface to equip the query-input with autocompletion. Also the application should deliver additional information about the indexes like description and a link to the definition of the underlying entity in the source registry.

Once there will be overlapping\footnote{i.e. different relations may  be defined for one data category in different relation sets} user-defined relation sets in the Relation Registry an additional input parameter will be required to \emph{explicitly restrict the selection of relation sets} to apply in the mapping function.

Also, use of \emph{other than equivalency relations will necessitate more complex logic in the query expansion and accordingly also more complex response of the SMC, either returning the relation types themselves as well or equip the list of indexes with some similarity ratio.}



\section{SMC on instance level}


\subsection{Mapping from strings to Entities}

Based on the textual values in the Metadata-descriptions find matching entities in selected Ontologies.

Identify related ontologies:
LT-World \cite{Joerg2010}

task:
\begin{enumerate}
\item  express MDRecords in RDF
\item  identify related ontologies/vocabularies (category -> vocabulary)
\item  use a lookup/mapping function (Vocabulary Alignement Service? CATCH-PLUS?)

%\fbox{ function lookup: Category x String -> ConceptualDomain}
\begin{eqnarray*}
lookup(Category, Literal) -> ConceptualDomain??
\end{eqnarray*}


Normally this would be served by dedicated controlled vocabularies, but expect also some string-normalizing preprocessing etc.
\end{enumerate} 


\subsection{Semantic Search}

Main purpose for the undertaking described in previous two chapters (mapping of concepts and entities) is to enhance the search capabilities of the MDService serving the Metadata/Resources-data. Namely to enhance it by employing ontological resources.
Mainly this enhancement shall mean, that the user can access the data indirectly by browsing one or multiple  ontologies, 
with which the data will then be linked. These could be for example ontologies of Organizations and Projects.

In this section we want to explore, how this shall be accomplished, ie how to bring the enhanced capabilities to the user.
Crucial aspect is the question how to deal with the even greater amount of information in a user-friendly way, ie how to prevent overwhelming, intimidating or frustrating the user.

Semi-transparently means, that primarily the semantic mapping shall integrate seamlessly in the interaction with the service, but it shall "explain" - offer enough information - on demand, for the user to understand its role and also being able manipulate easily.

?
Facets
Controlled Vocabularies 
Synonym Expansion (via TermExtraction(ContentSet))

\subsection{Linked Data - Express dataset in RDF}

Partly as by-product of the entities-mapping effort we will get the metadata-description rendered in RDF, linked with 
So theoretically we then only need to provide them "on the web", to make them a nucleus of the LinkedData-Cloud.

Practically this won't be that straight-forward as the mapping to entities will be a hell of a work.
But once that is solved, or for the subsets that it is solved, the publication of that data on the "SemanticWeb" should be easy.

Technical aspects (RDF-store?) / interface (ontology browser?)

defining the Mapping:
\begin{enumerate}
\item convert to RDF
translate: MDREcord -> [\#mdrecord \#property literal]
\item map: \#mdrecord \#property literal  -> [\#mdrecord \#property \#entity]
\end{enumerate}

\subsection{Content/Annotation}
AF + DCR + RR


\subsection{Visualization}
Landscape, Treemap, SOM

Ontology Mapping and Alignement / saiks/Ontology4 4auf1.pdf