source: SMC4LRT/chapters/userdocs_cleaned.tex @ 3776

Explore the \DUroletitlereference{Component Metadata Framework}

In \emph{CMD}, metadata schemas are defined by profiles, that are constructed out of reusable components  - collections
of metadata fields. The components can contain other components, and they can be reused in multiple profiles.
Furthermore, every CMD element (metadata field) refers via a PID to a data category to indicate unambiguously how the content of the field in a metadata description should
be interpreted (Broeder et al., 2010).

Thus, every profile can be expressed as a tree, with the profile component as the root node, the used components as intermediate nodes
and elements or data categories as leaf nodes, parent-child relationship being defined by the inclusion (\texttt{componentA -includes-> componentB}) or referencing (\texttt{elementA -refersTo-> datcat1}).The reuse of components in multiple profiles and especially also the referencing of the same data categories in multiple CMD elements leads to a blending of the individual profile trees into a graph (acyclic directed, but not necessarily connected).

SMC Browser visualizes this graph structure in an interactive fashion. You can have a look at the \href{examples.html}{examples} for inspiration.

It is implemented on top of wonderful js-library \href{https://github.com/mbostock/d3}{d3}, the code checked in \href{https://svn.clarin.eu/SMC/trunk/SMC}{clarin-svn} (and needs refactoring). More technical documentation follows soon.


\subsection{Data%
  \label{data}%
}

The graph is constructed from all profiles defined in the \href{http://catalog.clarin.eu/ds/ComponentRegistry/\#}{Component Registry}.
To resolve name and description of data categories referenced in the CMD elements
definitions of all (public) data categories from \href{http://dublincore.org}{DublinCore} and \href{http://www.isocat.org}{ISOcat} (from the \href{http://www.isocat.org/rest/profile/5.rdf}{Metadata Profile} {[}RDF{]} - retrieving takes some time!) are fetched. However only data categories used in CMD will get part of the graph. Here is a \href{smc_stats.html}{quantitative summary} of the dataset.

When inspecting the numbers, it is important to be aware of the occurrence expansion resulting from the reusability of the components.
So in an example, a component C has 2 subcomponents and is reused within one profile by two other components A and B, the resulting profile
will consist of (at least) 8 components (\texttt{{[}A, B, A/C, B/C, A/C/C1, A/C/C2, B/C/C1, B/C/C2{]}}), although only 5 distinct components are used.
The same goes for elements in reused components. In most cases it is indicated in the label, if the number reflect distinct items, or all (expanded) occurrences.

(Some of the) numbers in the statistics lead to a list of corresponding terms.
E.g. in the summary for a profile, clicking on the components-number lists all the components of given profile alphabetically.
Currently there are such lists for:
%
\begin{quote}
%
\begin{itemize}

\item \texttt{profile -> components}

\item \texttt{profile -> elements}

\item \texttt{profile -> data categories}

\item \texttt{data category -> profiles}

\end{itemize}

\end{quote}


\subsection{User Interface%
  \label{user-interface}%
}

The User interface is divided into 4 main parts:
%
\begin{description}
\item[{Index}] \leavevmode 
Lists all available Profiles, Components, Elements and used Data Categories
The lists can be filtered (enter search pattern in the input box at the top of the index-pane)
By clicking on individual items, they are added to the \DUroletitlereference{selected nodes} and get rendered in the graph pane

\item[{Main (Graph)}] \leavevmode 
Pane for rendering the graph.

\item[{Navigation}] \leavevmode 
This is the control panel governing the rendering of the graph. See below for available \hyperref[options]{Options}.

\item[{Detail}] \leavevmode 
In this pane, overall summary of the data is displayed by default,
but mainly the detail information about the selected nodes is listed here.

\end{description}


\subsection{Interaction%
  \label{interaction}%
}

Following data sets are distinguished wrt user interaction:
%
\begin{description}
\item[{all data}] \leavevmode 
the full graph with all profiles, components, elements and data categories and links between them.

Currently this amounts to roughly 2.000 nodes and 3.000 links

\item[{selected nodes}] \leavevmode 
nodes explicitely selected by the user (see below how to \hyperref[select-nodes]{select nodes})

\item[{data to show}] \leavevmode 
the subset of data that shall be displayed.

Starting from the selected nodes, connected nodes (and connecting edges)
are determined  based on the options (\texttt{depth-before}, \texttt{depth-after}).

\end{description}

The nodes are colour-coded by type:

\includegraphics[height=100px]{images/graph_legend.png}

\phantomsection\label{select-nodes}
There are multiple ways to select/unselect nodes:
%
\begin{description}
\item[{select from index}] \leavevmode 
by clicking individual items in the index list, the item will be \textbf{added} to the selected nodes

clicking on an already selected item unselects it

\item[{select in graph}] \leavevmode 
by clicking on a visible node in the graph, the node will be \textbf{added} to the selected nodes

clicking on an already selected node unselects it

\item[{select area in graph}] \leavevmode 
by dragging (hold mouse button down and pull) a rectangle in the graph pane, all nodes within that rectangle get selected
all other nodes will be unselected

\item[{unselect in detail pane}] \leavevmode 
clicking on an item in the detail pane unselects it

\item[{select in statistics}] \leavevmode 
as mentioned in \hyperref[data]{Data} (some) numbers in the statistics reveal a list of corresponding terms.
Clicking on these terms in the statistics page leads to the browser, with given term as selected node (and default settings)

\item[{select in statistics in the detail pane}] \leavevmode 
the numbers from statistics page are shown also in the detail pane for selected nodes.
Here, clicking on a term from these lists adds it to the graph, as a selected node.

\item[{mouseover}] \leavevmode 
on mouse over a node, all connected nodes to given node (and connecting links) within the visible sub-graph are highlighted
and all other nodes and links are faded

\item[{drag a node}] \leavevmode 
click and hold on a node, one can move the node around, however usually the layout is stronger
and puts the node back to its original position. Not so with the freeze-layout, that freezes all the nodes and lets you move them around freely

\end{description}


\subsection{Options%
  \label{options}%
}

The navigation pane provides following option to control the rendering of the graph:
%
\begin{description}
\item[{depth-before}] \leavevmode 
how many levels of connected ancestor nodes shall be displayed

\item[{depth-after}] \leavevmode 
how many levels of connected descendant nodes shall be displayed

\item[{link-distance}] \leavevmode 
approximate distance between individual nodes
(not exact, because it is just one of multiple factor for the layouting of the graph)

\item[{charge}] \leavevmode 
the higher the charge, the more the nodes tend to drift apart

\item[{friction}] \leavevmode 
factor for ``cooling down'' the layout, lower numbers (50-70) stabilize the graph more quickly,
but it may be too early, with higher numbers (95-100) the layout has more time/freedom to arrange,
but may get jittery

\item[{node-size}] \leavevmode 
N = all nodes have given diameter N;

usage = node is scaled based on how often the node appears in the complete dataset
i.e. often reused elements (like description or language) will be bigger

\item[{labels}] \leavevmode 
show/hide all labels
hiding the labels accelerates the rendering significantly, which may be an issue if more nodes are displayed.
irrespective of this option, on mouseover labels for all and only the highlighted nodes are displayed

\item[{curve}] \leavevmode 
straight or arc (better visibility)

\item[{layout}] \leavevmode 
There are a few layouting algorithms provided. They are all not optimal in any way, but most of the time, they deliver quite good results.
For different data displayed other algorithm may be more appropriate:
%
\begin{description}
\item[{force}] \leavevmode 
undirected layout, trying to spread the nodes in the pane optimally, equally in all directions
This is the underlying \href{https://github.com/mbostock/d3/wiki/Force-Layout}{layouting algorithm}. All the other layouts build on top of it, by just adding further constraints.

\item[{vertical-tree}] \leavevmode 
top-down layout respect the direction of the edges, children are always below the parents

\item[{horizontal-tree}] \leavevmode 
left-right layout respect the direction of the edges, children are always right to the parents
(at least they should be, currently, in certain configurations, the layout does not get the orientation for some links right)

\item[{weak-tree}] \leavevmode 
a layout that ``tends'' towards left to right arrangement, but not strictly so (experimental)

\item[{dot}] \leavevmode 
strict left to right reusing the x-positioning as determined by \href{http://www.graphviz.org/}{dot}
Arranges the nodes in strict ranks (typical for dot layout)
This is done in a separate preprocessing step for the whole graph, so the positioning may be suboptimal
for a given subgraph. The y-coordinate is approximated on the fly by the base algorithm.

\item[{freeze}] \leavevmode 
this is actually a ``no-layout'' - the nodes just stay fixed in their last position,
However, individual nodes still can be dragged around, so this can be used to adjust a few nodes for better legibility (or aesthetics),
but only when you start moving around inividual nodes, you will learn to appreciate the great (and tedious) work of the layouting algorithms,
so generally you want to try to play around with the other settings to achieve a satisfying result.

\end{description}

\end{description}


\subsection{Linking, Export%
  \label{linking-export}%
}

The navigation pane exposes a \textbf{link}, that captures the exact current state of the interface
(just the options and the selection, not the positioning of the elements),
so that it can be bookmarked, emailed etc.

Furthermore, there is the \textbf{download}, that allows to export the current graph as SVG.
This is accomplished without a round trip to the server, with a \href{https://groups.google.com/forum/?fromgroups=\#!topic/d3-js/aQSWnEDFxIc}{javascript trick}
serializing the svg as base64-data into the url (so you don't want to save (or see) the exported url).
But you can both, right click the link and {[}Save link as...{]}, or click on the link, which opens the SVG in a new tab
where you can view, resize, print and save it.
Employing this simple method also means, that there is no possibility to export the graph in PNG, PDF or any other format,
because this would require \href{http://d3export.cancan.cshl.edu/}{server-side processing}. (However this is a planned future enhancement.)


\subsection{Issues%
  \label{issues}%
}
%
\begin{description}
\item[{Performance}] \leavevmode 
Chrome is by far the fastest, followed by IE(9).
A serious performance degradation was observed for graphs above 200 nodes on Firefox.
Showing labels also significantly affects performance.

\item[{Bounds}] \leavevmode 
When the graph gets to big, it does not fit in the viewing pane.
This will be tackled soon (either scrollbars or applying boundaries). Meanwhile,
you can reduce the link-distance and charge parameters or change the layout.

\end{description}


\subsection{Plans and ToDos%
  \label{plans-and-todos}%
}

Substantial issues:
%
\begin{itemize}

\item Add information from \textbf{RelationRegistry} (relations between DatCats)

\item Blend in instance data from \textbf{MDRepository} (allow search on MDRepository)

\item graph operations (intersect, difference of subrgraphs)

\end{itemize}

Smaller enhancements of the user interface:
%
\begin{itemize}

\item select nodes by querying the names (e.g. show me all nodes with ``Access'' in their name)

\item option to show only selected types of nodes (e.g. only profiles and datcats)

\item detail-info on hover

\item full HTML-rendering of a node (Profile, Component)

\item backlinking from detail (e.g. view all the profiles a data category is used in by clicking on the number ('used in profiles')

\item store/export SVG/PDF/PNG-renderings of the graphs

\item add edge-weight: scale based on usage, i.e. how often appears the relation in the complete dataset
i.e. often reused combinations of components/elements will be nearer

\item allow to blend in further (private) CMD-profiles dynamically

\end{itemize}