wiki:VLO-Taskforce/Relations

Version 42 (modified by Jörg Knappen, 10 years ago) (diff)

--

VLO Taskforce (CLARIN-D)

  1. VLO Facets
  2. Exploiting ISOcat data categories
  3. Vocabularies
  4. Relationships
    1. HZSK
    2. BAS
    3. IDS-Mannheim
    4. Leipzig
    5. IMS Stuttgart
    6. Summary on Representation of Relationships
  5. Example Profiles
  6. Related Work

VLO Facets

Exploiting ISOcat data categories

Currently (January 2014) the VLO relies on two separate strategies for MD selection to populate its facets:

  1. ISOcat DCs associated with a MD field in the underlying CMDI profile
  2. a set of centre specific XPaths for CMDI instances to
    • explicitly select MD fields for inclusion in a facet that was not matched by a ISOcat DC (white list)
    • explicitly discard MD fields that were selected via their ISOcat DC (black list)

Ideally, strategy 1 should suffice for proper MD selection for the facets. To fix the current state of the MD instances and the VLO we will have to answer the questions:

  1. Does VLO rely on appropriate, i.e. sufficiently concrete defined ISOcat DCs? Obviously DCs like ​http://www.isocat.org/datcat/DC-2482 (language ID) or ​http://www.isocat.org/datcat/DC-2484 (language name) are semantically too vague and don't allow for the differentiation between the language a resource is written in or the language of an actor in case of transcribed recordings and so on.
    • Solution: Only use narrowly defined DCs for VLO facets.
    • Task: Evaluate the current mapping of ISOcat DCs to VlO facets
  2. Do CMDI profiles use sufficiently concrete ISOcat DCs?
    • Task: Create an overview of the DCs actually used by the centres.
    • Task: Evaluate the whitelist/blackclist XPaths with respect to why they are needed (to select vaguely defined DCs? to select MD fields that don't have a DC associated? anything else?) Axel Herold?: I will prepare such a list.
    • Task: Propose the adoption of container DCs for profiles that rely on vaguely defined DCs

Vocabularies

A first draft of recommended vocabularies (and in some cases components) was attached to this page (Gruppe Vokabular_v3.pdf (German)). There are some main problems that have to be adressed and that prevented a more complete recommendation:

  • missing precise specification of most facets
  • missing accordance between definition/vocabulary of elements in CMDI and linked datcats
  • many ISOcat definitions are problematic (to narrow, self-referential definitions, no explicit conceptual domain etc.)
  • a more powerful CMDI-based mechanism for specifying (external) vocabularies will only be part of CMDI 1.2

Relationships

The CMDI metadata framework provides for a generic mechanism to represent (directional) relationships between objects: <ResourceProxy?>. A <ResourceProxy?> specifies the <ResourceType?> (one of "Landing Page", "Resource", "Metadata") and the target <ResourceRef?> of a relationship, which (usually) contains a URL, sometimes a PID (as a special case of a URL). A <ResourceProxy?> is equipped with an id attribute, which can be and is being used for specifying further information, such as a descriptive, human readable anchor text and its semantics (partOf, versionOf, source, ...) in the component part of a CMDI record. Finally, for some kinds of relationships, there exist some dedicated elements in the <Resources> section of a CMDI record:

This huge, combinatorial design space for representing relationships has been creatively exhausted by the CLARIN-D Centers. As a consequence, the VLO is basically agnostic w.r.t. relationships.

In the following, some of the existing representations are analyzed, with the explicit goal of narrowing the design space and making some kinds of relationships more useful for the VLO. For some more information on the status of relationships in the discussion on CMDI 1.2 see also ​https://docs.google.com/spreadsheet/ccc?key=0Avyg_78eBoF4dFUxR2VpR01XRFEzSUVUb2tXcFduSXc&usp=sharing#gid=0

HZSK

HZSK represents all resources of a corpus in one CMDI record. Thus, the target of a relationship (<ResourceRef?>) has always type (ResourceType?) Resource. The individual resources are further specified in the component part of the CMDI record by referring to the <ResourceProxy?> via its id attribute.

Example:

​http://virt-fedora.multilingua.uni-hamburg.de/drupal/fedora/repository/cmdi:demo/cmdi/metadata.xml 

<ResourceProxy id="ACDIMB865D8-D9BA-7F9B-E652-D00D960850B4">
  <ResourceType mimetype="text/xml">Resource</ResourceType>
  <ResourceRef>http://hdl.handle.net/11858/00-248C-0000-000E-0181-F</ResourceRef>
</ResourceProxy>
...
<HZSKTranscription ComponentId="clarin.eu:cr1:c_1345561703658" ref="ID8392DD18-04C3-9DC7-A7F5-2FA8A3639EA4">
  <Name>Rudi Völler Wutausbruch</Name>
  <TranscriptionConvention>HIAT (simplified)</TranscriptionConvention>
  ...
</HZSKTranscription>

BAS

BAS represents the resources of a corpus by several CMDI records, and employs a variety of approaches to represent relationships:

  1. The relationship between the CMDI record for a corpus and its parts is specified explicitly in one direction by means of CMDI's built-in <isPartOf> element.
  2. Relationships with a target of type Resource are further specified in the component part of the CMDI record by referring to the <ResourceProxy?> via its id attribute
  3. Relationships between individual components, such as from <media-file> to <media-session-actor> are represented by referring to the target's id attribute as well.

Example:

​https://clarin.phonetik.uni-muenchen.de/BASRepository/Public/Corpora/ZIPTEL/ZIPTEL.2.cmdi.xml 

<ResourceProxy id="c_0000000001">
  <ResourceType mimetype="text/xml">Metadata</ResourceType>
  <ResourceRef>https://clarin.phonetik.uni-muenchen.de/BASRepository/Public/Corpora/ZIPTEL/0001.2.cmdi.xml</ResourceRef>
</ResourceProxy>

​http://catalog.clarin.eu/oai-harvester/cmdi-providers/harvested/results/cmdi/Bayerisches_Archiv_f_r_Sprachsignale/oai_BAS_repo_Corpora_ZIPTEL_0001.xml or ​https://clarin.phonetik.uni-muenchen.de/BASRepository/Public/Corpora/ZIPTEL/0001.2.cmdi.xml 

<ResourceProxy id="r_0000000001">
  <ResourceType mimetype="audio/raw">Resource</ResourceType>
  <ResourceRef>https://clarin.phonetik.uni-muenchen.de/BASRepository/Corpora/ZIPTEL/0001/z10001z2.dea</ResourceRef>
</ResourceProxy>
...
<IsPartOfList>
   <IsPartOf>https://clarin.phonetik.uni-muenchen.de/BASRepository/Public/Corpora/ZIPTEL/ZIPTEL.2.cmdi.xml</IsPartOf>
</IsPartOfList>
...
<media-file actor-ref="s_0000000001" ref="r_0000000001">
  <Type>audio</Type><Quality>3</Quality>
  <RecordingConditions>un-supervised answering of a question prompted via telephone</RecordingConditions>
...
</media-file>
...
<media-session-actor id="s_0000000001">
  <Role>question answering</Role> 
  <Name>unspecified</Name>
  <FullName>unspecified</FullName>
  <Code>0001</Code>
  ...
</media-session-actor>

IDS-Mannheim

IDS-Mannheim represents resources by several CMDI records, and employs a variety of approaches to represent relationships:

  1. The first version of the historical newspaper corpus MKHZ represents relationships by <ResourceProxy?> as well as OLAC-Dcmi-Terms elements such as <hasPart>, where both point to a PID (see for example ​http://repos.ids-mannheim.de/fedora/objects/clarind-ids:mkhz.000000/datastreams/CMDI/content)
  2. The second version of the historical newspaper corpus represents relationships by <ResourceProxy?> and further specifies the semantics (and anchor text) of the relationship in the component part of the relationship (see for example ​http://repos.ids-mannheim.de/fedora/objects/clarin-ids:mkhz1.00000/datastreams/CMDI/content). The underlying conceptual model is depicted in the figure below.
  3. Relationships in the corpora of spoken language are represented by ResourceProxy?'s only, and partOf relationships are further specified by means of CMDI's built-in <IsPartOf?> element (see for example ​http://repos.ids-mannheim.de/fedora/objects/clarin-ids:folk.FOLK_S_00248.cmdi/datastreams/CMDI/content)

conceptual model of MKHZ representation (small)

Leipzig

Leipzig's approach to representing relationships is similar to IDS (and BAS) Option b. The differences are as follows:

  1. For each relationship there exists a separate Component, which already has a built-in attribute ref of type idrefs.
  2. The description of a relationship is structured rather than just a simple anchor text.
  3. Inverse relationships are not represented explicitly.

Example: 

<ResourceProxy id="ulei-11858-00-229C-0000-0001-B06F-3-component-dataprovider-1">
  <ResourceType mimetype="text/xml">Metadata</ResourceType>
  <ResourceRef>http://hdl.handle.net/11858/00-229C-0000-0001-B06F-3@type=dataprovider&id=1</ResourceRef>
</ResourceProxy>
...
<LCC_DataProvider ComponentId="clarin.eu:cr1:c_1381926654509" ref="ulei-11858-00-229C-0000-0001-B06F-3-component-dataprovider-1">
  <Id>1</Id>
  <Name>LCC data provider "www.shortnews.de" in resource with identifier 11858/00-229C-0000-0001-B06F-3</Name>
  <Description xml:lang="eng">Data provider of the Leipzig Corpora Collection: www.shortnews.de</Description>
</LCC_DataProvider>

IMS Stuttgart

This is a list of the relations which we would like to represent in the CMDI records, and how we deal with them so far. Due to the fact, that we were not sure about recommendations how to use the 'Resources' component, we have not applied it yet, but are of course interested in using it, so that some of the relations might also be exploited in the VLO:

    • Relation type: trained model -- data set on which the model was trained
    • Current approach: Optional CMDI component 'BasedOn' as part of the profile to describe tool components (CMDI component not in public section yet).
    • Examples:
      • Dutch parameter file for TreeTagger --> trained on Eindhoven corpus
      • German grammar for BitPar --> extracted from Tiger treebank
    • Relation type: versioning
    • Current approach: Element 'Version' in CMDI component 'GeneralInfo' (clarin.eu:cr1:c_1290431694495) and where applicable a common part in the 'ResourceName' to implicitly relate different versions.
    • No examples yet, but this will change in future. Affects all types of resources: corpora, lexicons, tools, web services, ...
    • Relation type: tool -- web service
    • We keep getting mails asking for information and publications on the tools 'behind' the web services. This also affects the action item regarding the CMDI profiles for web services.

Further notes:

  • Making these relations explicit will also mean more maintenance effort to keep the md-records up to date, thus we should discuss where this will make sense and where it creates too much overhead.
  • Some relationships are between 'clearly distinct' resources: 'Source' was mentioned in the Tischvorlage, others might include:
    • 'A is based on B' e.g. when a trainable component (parameterfile, language model, ...) has been trained on a specific corpus, or a collection of technical terms has been extracted based on the frequencies of a specific corpus resource (see relation type b.)
    • 'A can be used with B', e.g. when a tool can use an (additional) knowledge base (lexicon, language model, ...), or a corpus is encoded for a specific query engine (see relation type a.)
    • 'A was used for the creation of B', e.g. when manual annotations were added with the help of a specific system, ...
    At the moment some of these relationships can be expressed in the component section of the CMDI-records (e.g. 'Derivation', 'Source') but are not exploited in the VLO.

Summary on Representation of Relationships

The conceptual model for relationships is fairly simple:

A relationship

  1. relates two or more resources,
  2. has a semantic type (and possibly semantic roles for the related resources, such as source/target)
  3. has some human readable description, ranging from simple text to sth. more structured.

While the built-in mechanisms in CMDI can not express all the aspects of the conceptual model, data providers have found ways to combine CMDI's built-in mechanisms with the full flexibility provided by CMDI components to encode this conceptual model or some variants of it.

This initial attempt of a summary can probably not do justice to the full solution space.

(1) Relationships between Internal Resources

These relationships are established between internal resources.

(1.1) Specified by ResourceRelationList?

For a discussion of this option see the CMDI 1.2 Issue https://trac.clarin.eu/wiki/CMDI%201.2/Resource%20proxies/ResourceRelation (e.g. solution 2).

I am currently not aware of any data provider in CLARIN-D using this option.

(1.2) Specified in the Components Part

For an example see https://trac.clarin.eu/wiki/%20VLO%20Taskforce#BAS, second example, which establishes a relationship media-file between a Resource with id=r_0000000001 and an Actor with id=s_0000000001 by way of a component (clarin.eu:cr1:c_1336550377512)

The Resource is listed in ResourceProxyList?, the Actor is specified as an Element in the component part.

The semantic type of this relationship is not specified explicitly, i.e. the component media-file is not associated with a data category.

(2) Relationships from a Resource to other Resources

As the title indicates, these relationships are established from a resource of type Metadata (a CMDI record) to other Resources (of Type Metadata or Resource).

(2.1) ResourceProxyList?

ResourceProxyLists? are the most straightforward way to express such relationships. However, they do not provide for any direct means to specify (b) the semantic type of a relationship or (c) a human readable description.

There seems to be an implicit assumption among some CMDI illuminati that the semantic type of such relationships is sth. like hasPart. For representing the inverse of this assumption, CMDI currently provides isPartOf as a built-in.

(2.2) ResourceProxyList? + Component for further information

It has become evident that there is a need to represent many more kinds of relationships between Resources than just hasPart/isPartOf.

One popular profile (OLAC_DcmiTerms) lists the following relationships, with the according dcr:datcat:

At least one data provider (IDS-Mannheim) thus has chosen to represent the additional semantics of a relationships in the component part, by reusing the semantics of the undlerlying dcr:datcat categories. See for example: ​http://hdl.handle.net/10932/00-01B8-AE41-41A4-DC01-5

This approach represents (a) the reference to the resource via a (not supported) attribute ref pointing to the id of a resourceProxy, (b) the semantic role of the relationship via the dcr:datcat attribute in the underlying component specification, and (c) the human readable description via the content of the element.

The description of (planned) relationships at IMS Stuttgart (https://trac.clarin.eu/wiki/%20VLO%20Taskforce#IMSStuttgart) warrants some further analysis, but generally their overall design pattern seems to be inline with the design pattern followed in this section.

(2.3) ResourceProxyList? + isPartOfList

TBD: example: ​http://hdl.handle.net/10932/00-01B8-EDCB-0B8B-D501-C

(3) Summary

There is an obvious need for representing and using more information about relationships than currently supported by the built-in mechanisms in CMDI.

The key question is whether this need is to be fulfilled by (a) extending built-in encodings in CMDI ("easy" for the VLO) or by (b) using CMDI's extensibility features (components + iso/dcr:cat) in a more principled manner (more difficult for the VLO).

For a rather elaborate discussion of the pros and cons of (a) vs. (b) see https://trac.clarin.eu/wiki/CMDI%201.2/Resource%20proxies/ResourceRelation

Example Profiles

Related Work

​DataCite has similar goals as CLARIN: sharing and persistently identifying research data. For the description of it uses a fixed schema, which is somewhat inspired by Dublin Core. In terms of CMDI, the schema comprises both, facets and relationships. For more information see the ​DataCite Schema Repository.

Attachments (7)

Download all attachments as: .zip