wiki:ComponentVersioning

The purpose of this page is to come to a design for an extension of the ComponentRegistry that will enable versioning and deprecation of components and profiles.

Component versioning will be taken up as a part of CMDI 1.2, see CMDI 1.2/Lifecycle management for more information!

  1. Component versioning
    1. Definitions
    2. Introduction/Use case
    3. Design
      1. Status
      2. Views (workspaces and registries)
      3. Discussion on possible actions
    4. Implementation
      1. Storage / internal representation
      2. Client information
    5. Considerations for tools further down the chain
      1. Metadata editing
      2. Search & catalogue tools
    6. Notes
    7. Discussion
    8. Related tickets

Component versioning

Definitions

  • Component refers to both components and profiles as specified by the general component schema that exist in the ComponentRegistry;
  • Versioning refers to the precursor/successor relation that may exists between two components. It does not imply a structural or functional extension or inheritance, i.e. the structure of the precursor component does not put any constraints on the structure of the successor relation; no assumptions about the structure of the' one can be made on basis of the structure of the other. Versioning does not imply deprecation (see below) of the precursor.
  • Deprecation refers to a state in which a component is no longer 'advertised' by the registry (it will not appear in the list of published components) and will be explicitly marked as such to the client. However it will still be accessible by its URI so that instances based on deprecated components will remain valid. Apart from this, the properties of published components apply. Deprecation can optionally be combined with versioning.

Introduction/Use case

The goal of designing components and profiles in the ComponentRegistry is to eventually make them available in the public space so that everyone can use them as a basis for metadata creation. However, once published a component gets 'frozen' so that all instantiations can rely on it not changing and thus are guaranteed to retain their validity. Also, after a certain 'cooling down' period, published components cannot be removed from the public space (except by administrators).

Robust though this practice may be, it leads to issues when changes in the domain, or simply new insights, need to be incorporated in the component. It is easy to create a new component based on an existing one, extend it and finally make it public. But then one would like to communicate to the users of the component that they should use the new component for new metadata (possibly even convert existing metadata) instead of the old one. However, there are a few problems:

  • There is no reliable way to find out who is instantiating specific components, so the users are unknown and cannot simply be informed
  • The old component cannot be deleted, so new users might accidentally start using the old one
  • All of this is very informal, and requires a lot of work with few guarantees

A formal way of deprecating components and specifying versioning information would solve these issues. The owner of a component can simply:

  1. Create a new component (typically copying the existing component as a basis)
  2. Edit until satisfied
  3. Publish into public space
  4. Deprecate the 'precursor' component
  5. Designate the new component as 'successor' to the old one

The ComponentRegistry can then provide this information to all clients that request the deprecated component, and remove it from the public list. Clients should of course still be able to use the deprecated component since it is not always possible to upgrade.

Design

Status

For each component the following will be specified and communicated to the client:

  • A status indication, exactly one value out of
    • development (may still be edited by owner(s). Only listed if marked for review)
    • production (should be listed publicly, read-only to everyone)
    • deprecated (no longer in public listing, queryable and read-only to everyone)
  • A optional indication of the direct successor of the component if it is deprecated

Furthermore, each component will be owned by either a user or a group (see #143) regardless of its status. The Component Registry allows development components to be marked for review, which includes them in a public list of components under review.

Only owners of a component (i.e. the user that owns it or any member of the group that owns it) will be allowed to change the status of a component (i.e. development -> production and production -> deprecated).

Views (workspaces and registries)

The above will lead to a number of possible views on the component space, which conceptually correspond with what is called a 'registry' (ComponentRegistry interface).

The following views should be made available through the rest service:

  • Personal workspace
    • e.g. /rest/registry/personal/components
    • includes one user's development components (identical to current 'user space'), including those marked for review
  • Group workspace
    • e.g. /rest/registry/group/1/components
    • includes one group's development components, including those marked for review
  • Review registry
    • e.g. /rest/registry/review/components
    • includes any user's or group's components marked for review (therefore overlaps with the personal and group workspaces)
  • Public registry
    • e.g. /rest/registry/public/components
    • includes all production components that have not been deprecated (identical to current public registry)
  • Deprecated registry (morgue)
    • e.g. /rest/registry/deprecated/components
    • includes all deprecated components

The ComponentRegistry web service (and as a result this will also apply to the Flex-based component browser) will not display deprecated components in the currently existing public or private listings (e.g. ​/registry/profiles). However, they will still be available by their ID (like ​/registry/profiles/clarin.eu:cr1:p_1297242111880).

There will be a call to get the successor (and possibly also recursive all successors or at least the latest successor) of a specific component (see XML example below).

Both the XML representation of a component and the XSD that results from the comp2schema transformation should contain deprecation and versioning information in some form (see below for a proposal).

Discussion on possible actions

To decide: will actions on these views be allowed or can the status of individual components only be modified explicitly? The only possible status change currently is to publish, which is done by posting a component to /components/{componentId}/publish which is not as RESTful as it could be.

There are are two alternative ways of achieving this in the proposed situation with more that two states:

  • Change status through PUT update on the component itself
  • Change status by POSTINGting resources into views

The two alternatives are explained in the sections below.

Change status through update on the component itself

The status is part of the item description and/or component spec (see #146), so simply PUTting an update to an item description (through its persistent /components/{componentId} URI) can be used to change the state. For an existing component, e.g.: 

<componentDescription xmlns:ns2="http://www.w3.org/1999/xlink">
    <id>clarin.eu:cr1:c_1311927752326</id>
    <description>Component contains information about the availability and accessibility of a resource</description>
    <name>Access</name>
    (...)
    <status>Development</name>
</componentDescription>

...one could PUT an update to /rest/registry/components/{componentId}, e.g.: 

<componentDescription xmlns:ns2="http://www.w3.org/1999/xlink">
    <id>clarin.eu:cr1:c_1311927752326</id>
    <description>Component contains information about the availability and accessibility of a resource</description>
    <name>Access</name>
    (...)
    <status>Public</name>
</componentDescription>

...to publish the draft.

The change would automatically get reflected in the views as they are all logical.

Change status by POSTING components into views

Alternatively, posting and putting components into the views described above could be used as an indirect way of modifying the status of a component:

  • Adding a component to the personal workspace will make it a development component owned by the user
  • Adding a component to the group workspace will make it a development component owned by the group
  • Moving an existing component from the personal workspace to the group workspace will transfer the ownership from the individual user to the group
  • Moving an existing component to the review registry will make it a review component keeping the original owner
  • Moving an existing component to the public regisry means to publish it; after publication it cannot be modified
  • Moving an existing component to the deprecated registry will mean to deprecate it, so it will no longer appear in the public registry and remains frozen

the move action would be implemented through a POST on the target registry, which would implicitly remove the specified component from the source registry. While the views remain logical, they would function as 'virtual resources' that actually allow some CRUD operations.

Preferred solution

In either solution, the canonical component URI would always have to keep adhering to the pattern /registry/components/{componentId} or /registry/profiles/{profileId} (we don't want the URI to change simply because the state changes). This would be slightly awkward in the case of the second solution (POST to views) where no resource actually gets created as a result. Also, this solution is less direct than the 'PUT update component' solution so the former seems preferable.

Implementation

Storage / internal representation

The ComponentRegistry web service has a PostgreSQL back-end in which it stores both the component specifications in their verbatim XML formatting and 'component metadata' (referred to as 'descriptions' to avoid ambiguity) containing fields such as 'creator' and 'group'. The 'status' property can be added to this. To accommodate the successor relations, an additional table has to be added simply linking precursors and successors, and optionally storing a comment on each relation.

Client information

Because most clients will consume the XSD transformation of components, the status and versioning information will have to be represented in these profile schemata. It can be provided within an xs:appinfo element. This information can be taken from two proposed new optional header elements in the component specification: Status, StatusComment? and Successor. The contents of these will come from the database.

Example: 

<CMD_ComponentSpec isProfile="true">
   <Header>
      <ID>clarin.eu:cr1:p_1289827960126</ID>
      <Name>LrtInventoryResource</Name>
      <Description>Resources as stored before in the CLARIN LRT inventory</Description>
      <Status>deprecated</Status>
      <StatusComment>The following fields were missing: actor age, content language</StatusComment>
      <Successor>http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/profiles/clarin.eu:cr1:p_1289827960126</Successor>
   </Header>

in the XSD this would be transformed to: 

<xs:schema xmlns:cmd="http://www.clarin.eu/cmd/"
      xmlns:xs="http://www.w3.org/2001/XMLSchema"
      xmlns:dcr="http://www.isocat.org/ns/dcr"
      xmlns:ann="http://www.clarin.eu"
      targetNamespace="http://www.clarin.eu/cmd/"
      elementFormDefault="qualified">
   <xs:import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="http://www.w3.org/2001/xml.xsd"/>
   <xs:annotation>
      <xs:appinfo>
         <cmd:Status>deprecated</cmd:Status>
         <cmd:StatusComment>The following fields were missing: actor age, content language</cmd:StatusComment>
         <cmd:Successor>http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/profiles/clarin.eu:cr1:p_1289827960126/successors</cmd:Successor>
      </xs:appinfo>
   </xs:annotation>

Considerations for tools further down the chain

Metadata editing

Deprecated versions of profiles should not be advertised to the user (e.g. should not appear in the profiles list in Arbil) but there should be no restriction in using them (i.e. the can still be added by their schema URL) and existing metadata will be untouched,. The metadata editor however could indicate the deprecated status of used profiles, including (non-obtrusive) notifications about the status of the profile the metadata being worked on is based on. For example, the icon shown for the metadata documents could reflect the state of the profile. Similarly, the list of active profiles in Arbil could mark those profiles in the list that are marked as deprecated. In case a successor is available, the editor should notify the user about that as well and ideally offer to load that profile instead (or in addition). Automatic upgrading to the new version should not happen, but assisting the user in attempting a migration could be considered (here the user should be warned about potential implications on the semantics of the existing values as a result!).

Search & catalogue tools

Search tools that explicitly distinguish between profiles in the user interface could apply clustering with respect to different versions of the same profile (traversing the succession chain). In those cases where profile names are shown, it could make sense to show the status of the profile (e.g. under development, deprecated) if this is considered to be of interest to the user.

Notes

  • A specific version of a profile will always keep its unique profile ID
  • Only owners (and admins) can change the status of a component (as is the case with publishing now)
  • Deprecation and succession will require a reason (for deprecation) or comment (for new version)

Discussion

  • Should 'bi-directional' versioning information be provided or just successors? (should a successor component refer to its precursor?)
    • Consensus seems to be: no
  • What to do with many-to-many relations in versioning? (merging and branching of components)
    • Consensus seems to be: no special support, this is only relevant in the context of inheritance
  • ...

Related tickets

Ticket Component Summary Milestone
#142 ComponentRegistry Component and profile versioning/deprecation
#143 ComponentRegistry Shared private components (groupspace) ComponentRegistry-1.14
#146 ComponentSchema Add a status field (development, production, deprecated) CMDI 1.2

Last modified 11 years ago Last modified on 10/25/13 12:07:04