Changeset 2696 for SMC4LRT/chapters/SMC.tex

Timestamp:

03/13/13 18:06:34 (11 years ago)

Author:

vronk

Message:

added notes from Menzo about DCR CLAVAS interaction

File:

• SMC4LRT/chapters/SMC.tex (modified) (8 diffs)

SMC4LRT/chapters/SMC.tex

@@ +1 @@
+
 \chapter{Semantic Mapping Component}
 
 
-
-\section{?? DataModel}
+\section{Data Model}
 
 Terms ? 
@@ -11 +11 @@
 
 
-\section{Semantic Mapping on concept level}
-
-merging the pieces of information provided by those,
-offering them semi-transaprently to the user (or application) on the consumption side.
-
-a module of the Component Metadata Infrastructure performing semantic mapping on search indexes. This  builds the base for query expansion to facilitate semantic search and enhance recall when querying the Metadata Repository.
+\subsection{CMD namespace}
+Describe the CMD-format?
+
+
+\subsection{DCR in SKOS}
+\label{dcr-skos}
+Describe the mapping from DCR into SKOS
+
+DCR recognizes following types of data categories:
+simple, complex: closed, open, constrained, (container)?
+
+\begin{figure*}[!ht] 
+\begin{center}
+\includegraphics[width=0.7\textwidth]{images/dc_types}
+\end{center}
+\caption{Data Category types}
+\end{figure*}
+\todo{cite:  ISOcat introduction at CLARIN-NL Workshop}
+
+The export to CLAVAS-SKOS only considers/regards closed and simple DCs from the metadata profile are exported. 
+A closed DC maps to a concept scheme and a simple DC to a SKOS concept in such a concept scheme. 
+However it needs to be yet assessed how useful this approach is. In the metadata profile
+there are many closed DCs with small value domains. How useful are those
+in CLAVAS? 
+Originally, the vocabulary repository has been conceived to manage rather large and complex value domains,
+that do not fit easily in the DCR data-model.
+Therefore a threshold seems sensible, where only value domains with more
+then 20, 50 or 100 values are exported.
+
+Open or constrained DCs are not exported as they don't provide anything to a vocabulary. \todo{cite: Menzo2013-03-12 mail}
+However, they can become users of a CLAVAS vocabulary. Actually, providing vocabularies for constrained but large and complex conceptual domains is the main motivation for the vocabulary repository.
+
+Currently (before integration of VAS and DCR), the only possibility to constrain the value domain of a data category
+is by the means a XML Schema provides, like a regular expression. So for the data category \concept{languageID DC-2482}
+the rule looks like:
+\lstset{language=XML}
+\begin{lstlisting}
+        <dcif:conceptualDomain type="constrained">
+                <dcif:dataType>string</dcif:dataType>
+                <dcif:ruleType>XML Schema regular expression</dcif:ruleType>
+                <dcif:rule>[a-z]{3}</dcif:rule>
+        </dcif:conceptualDomain>
+\end{lstlisting}
+
+A current proposal by Windhouwer\todo{cite: Menzo2013-03-12 mail} for integration with CLAVAS foresees following extension:
+
+\begin{lstlisting}
+        <clavas:vocabulary href="http://my.openskos.org/vocab/ISO-639" type="closed"/>
+\end{lstlisting}
+
+\code{@href} points to the vocabulary. Actually a PID should be used in the context
+of ISOcat, but it is not clear how persistent are the vocabularies. This may pose a problem as part of DC specification may now have a different persistency then the core.
+
+\code{@type} could be \code{closed} or \code{open}. \code{closed}: only values in the vocabulary are
+valid. \code{open}: the values in the vocabulary are hints/preferred values. Basically the DC itself is then open.
+
+This would yield a definition of the conceptualDomain for the data category as follows:
+ 
+\lstset{language=XML}
+\begin{lstlisting}
+  <dcif:conceptualDomain type="constrained">
+     <dcif:dataType>string</dcif:dataType>
+     <dcif:ruleType>XML Schema regular expression</dcif:ruleType>
+     <dcif:rule>[a-z]{3}</dcif:rule>
+  </dcif:conceptualDomain>
+  <dcif:conceptualDomain type="constrained">
+     <dcif:dataType>string</dcif:dataType>
+     <dcif:ruleType>CLAVAS vocabulary</dcif:ruleType>
+      <dcif:rule>
+         <clavas:vocabulary href="http://my.openskos.org/vocab/ISO-639" type="closed"/>
+      </dcif:rule>
+  </dcif:conceptualDomain>
+\end{lstlisting}
+
+I.e. the new rule pointing to the vocabulary would be \emph{added}, so that tools that don't support CLAVAS
+lookup but are capable of XSD/RNG validation, can still use the regular expression based definition.
+
+ 
+\begin{note}
+
+\noindent
+something similar for the link to an EBNF grammar in SCHEMAcat: 
+
+%\begin{lstlisting}
+\begin{verbatim}
+      <scr:valueSchema 
+               xmlns:scr="http://www.isocat.org/ns/scr" 
+               pid="http://hdl.handle.net/1839/00-SCHM-0000-0000-004A-A" 
+               type="ISO 14977:1996 EBNF"/>
+\end{verbatim}
+%\end{lstlisting}
+\end{note}
+
 
 
@@ -28 +115 @@
 
 We distinguish two types of smcIndexes: (i) \emph{dcrIndex} referring to data categories and (ii) \emph{cmdIndex} denoting a specific
-"CMD-entity", i.e. a metadata field, component or whole profile defined within CMD. The \textit{cmdIndex} can be interpreted as a XPath into the instances of CMD-profiles. In contrast to it, the \textit{dcrIndexes} are generally not directly applicable on existing data, but can be understood as abstract indexes referring to well-defined concepts -- the data categories -- and for actual search they need to be resolved to the metadata fields they are referred by. In return one can expect to match more metadata fields from multiple profiles, all referring to the same data category.
+``CMD-entity'', i.e. a metadata field, component or whole profile defined within CMD. The \textit{cmdIndex} can be interpreted as a XPath into the instances of CMD-profiles. In contrast to it, the \textit{dcrIndexes} are generally not directly applicable on existing data, but can be understood as abstract indexes referring to well-defined concepts -- the data categories -- and for actual search they need to be resolved to the metadata fields they are referred by. In return one can expect to match more metadata fields from multiple profiles, all referring to the same data category.
 
 These two types of smcIndex also follow different construction patterns:
@@ -63 +150 @@
 
 
-\subsection{Function}\label{method}
+\subsection{Query language}
+CQL?
+
+
+\section{Semantic Mapping on concept level}
+
+merging the pieces of information provided by those,
+offering them semi-transaprently to the user (or application) on the consumption side.
+
+a module of the Component Metadata Infrastructure performing semantic mapping on search indexes. This  builds the base for query expansion to facilitate semantic search and enhance recall when querying the Metadata Repository.
+
+
 In this section, we describe the actual task of the proposed application -- \textbf{mapping indexes to indexes} -- in abstract terms. The returned mappings can be used by other applications to expand or translate the original user query, to match elements in other schemas.
 \footnote{Though tightly related, mapping of terms and query expansion are to be seen as two separate functions.}
 % \footnote{This primary usage of SMC for work with user-created query strings explains the need for human-readability of the indices.}
 
-
-\subsubsection*{Initialization}
-
-First there is an initialization phase, in which the application fetches the information from the source modules (cf. \ref{components}). All profiles and components from the Component Registry are read and all the URIs to data categories are extracted to construct an inverted map of data categories:
-\newline
-
-\textit{datcatURI $\mapsto$ profile.component.element[]}
-\newline
-
-The collected data categories are enriched with information from corresponding registries (DCRs), adding the verbose identifier, the description and available translations into other working languages. %, usable as base for multi-lingual search user-interface.
-
-Finally relation sets defined in the Relation Registry are fetched and matched with the data categories in the map to create sets of semantically equivalent (or otherwise related) data categories. 
-
-\subsubsection*{Operation}
 In the operation mode, the application accepts any index (\textit{smcIndex}, cf. \ref{indexes}) and returns a list of corresponding indexes (or only the input index, if no correspondences were found):
 \newline
@@ -117 +201 @@
 \newline 
 
-(3) \emph{container data categories} -- further expansions will be possible once the container data categories \cite{SchuurmanWindhouwer2011} will be used. Currently only fields (leaf nodes) in metadata descriptions are linked to data categories. However, at times, there is a need to conceptually bind also the components, meaning that besides the "atomic" data category for \texttt{actorName, there would be also a data category for the complex concept \texttt{Actor}.} 
+(3) \emph{container data categories} -- further expansions will be possible once the container data categories \cite{SchuurmanWindhouwer2011} will be used. Currently only fields (leaf nodes) in metadata descriptions are linked to data categories. However, at times, there is a need to conceptually bind also the components, meaning that besides the ``atomic'' data category for \texttt{actorName, there would be also a data category for the complex concept \texttt{Actor}.} 
 Having concept links also on components will require a compositional approach to the task of semantic mapping, resulting in:
 \newline 
@@ -141 +225 @@
 \subsection{Mapping from strings to Entities}
 
-Based on the textual values in the Metadata-descriptions find matching entities in selected Ontologies.
+Find matching entities in selected Ontologies based on the textual values in the metadata records.
+
 
 Identify related ontologies:
@@ -162 +247 @@
 
 
-\subsection{Semantic Search}
-
-Main purpose for the undertaking described in previous two chapters (mapping of concepts and entities) is to enhance the search capabilities of the MDService serving the Metadata/Resources-data. Namely to enhance it by employing ontological resources.
-Mainly this enhancement shall mean, that the user can access the data indirectly by browsing one or multiple  ontologies, 
-with which the data will then be linked. These could be for example ontologies of Organizations and Projects.
-
-In this section we want to explore, how this shall be accomplished, ie how to bring the enhanced capabilities to the user.
-Crucial aspect is the question how to deal with the even greater amount of information in a user-friendly way, ie how to prevent overwhelming, intimidating or frustrating the user.
-
-Semi-transparently means, that primarily the semantic mapping shall integrate seamlessly in the interaction with the service, but it shall "explain" - offer enough information - on demand, for the user to understand its role and also being able manipulate easily.
-
-?
-Facets
-Controlled Vocabularies 
-Synonym Expansion (via TermExtraction(ContentSet))
-
-\section{Linked Data - Express dataset in RDF}
+\subsection{Linked Data - Express dataset in RDF}
 
 Partly as by-product of the entities-mapping effort we will get the metadata-description rendered in RDF, linked with 
-So theoretically we then only need to provide them "on the web", to make them a nucleus of the LinkedData-Cloud.
-
-Practically this won't be that straight-forward as the mapping to entities will be a hell of a work.
-But once that is solved, or for the subsets that it is solved, the publication of that data on the "SemanticWeb" should be easy.
+So theoretically we then only need to provide them ``on the web'', to make them a nucleus of the LinkedData-Cloud.
+
 
 Technical aspects (RDF-store?) / interface (ontology browser?)
@@ -201 +268 @@
 \end{figure*}
 
-\subsection{Content/Annotation}
+
+\section{Semantic Search}
+
+Main purpose for the undertaking described in previous two chapters (mapping of concepts and entities) is to enhance the search capabilities of the MDService serving the Metadata/Resources-data. Namely to enhance it by employing ontological resources.
+Mainly this enhancement shall mean, that the user can access the data indirectly by browsing one or multiple  ontologies, 
+with which the data will then be linked. These could be for example ontologies of Organizations and Projects.
+
+In this section we want to explore, how this shall be accomplished, ie how to bring the enhanced capabilities to the user.
+Crucial aspect is the question how to deal with the even greater amount of information in a user-friendly way, ie how to prevent overwhelming, intimidating or frustrating the user.
+
+Semi-transparently means, that primarily the semantic mapping shall integrate seamlessly in the interaction with the service, but it shall ``explain'' - offer enough information - on demand, for the user to understand its role and also being able manipulate easily.
+
+?
+Facets
+Controlled Vocabularies 
+Synonym Expansion (via TermExtraction(ContentSet))
+
+\subsection{Query Expansion}
+
+
+\section{Semantic Mapping in Metadata vs. Content/Annotation}
 AF + DCR + RR
 
 
-\subsection{Visualization}
-Landscape, Treemap, SOM
-
-Ontology Mapping and Alignement / saiks/Ontology4 4auf1.pdf
-
+
+