This page is a subpage of CMDI 1.2

Component lifecycle management in CMDI 1.2

An executive summary is available at CMDI 1.2/Lifecycle management/Summary

The issue

The issue has been described and explored on the page ComponentVersioning. The issue description below is based on the one found on that page.

Once published a component gets 'frozen' so that all instantiations can rely on it not changing and thus are guaranteed to retain their validity. Also, after a certain 'cooling down' period, published components cannot be removed from the public space (except by administrators). Robust though this practice may be, it leads to issues when changes in the domain or new insights need to be incorporated in the component. It is easy to create a new component based on an existing one, extend it and finally make it public. But then one would like to communicate to the users of the component that they should use the new component for new metadata (possibly even convert existing metadata) instead of the old one. However, there are a few problems:

• There is no reliable way to find out who is instantiating specific components, so the users are unknown and cannot simply be informed
• The old component cannot be deleted, so new users might accidentally start using the old one
• All of this is very informal, and requires a lot of work with few guarantees

A formal way of deprecating components and specifying versioning information would solve these issues. In such a workflow, the owner of a component can simply:

0. Create a new component (e.g. by copying the existing component as a basis)
1. Edit until satisfied
2. Publish into public space
3. Deprecate the 'precursor' component
4. Designate the new component as 'successor' to the old one

This information needs to be communicated to the client that request the deprecated component, and remove it from the public list. Clients should of course still be able to use the deprecated component since it is not always possible to upgrade.

Proposed solution

Based on the proposal as worked out in the page ComponentVersioning.

The component specification needs to be extended with a number of header elements that contain the versioning information. The header elements to be added are described in the following subsections. Because the header gets embedded into an 'appinfo' element in the schema derived from profile specifications, the state and versioning information is available from the schema as well, which is convenient for tools that (primarily) make use of the schema.

In addition to these extensions to the component model, the infrastructure needs to be adapted to implement a workflow for changing lifecycle statuses and the consequences of such changes. More specifically, the CMDI Component Registry will have to manage the status of each component and discourage users (both metadata modellers and creators) from using components or profiles that carry a non-production (i.e. development or deprecated) state. Details of the required changes are given below under 'centre impact'.

Status header

/CMD_ComponentSpec/Header/Status

This mandatory element indicates the status of the component at time of retrieval. It has exactly one value out of the following:

• development (may still be edited by owner(s); only listed if marked for review)
• production (should be listed publicly, read-only to everyone)
• deprecated (no longer in public listing, queryable and read-only to everyone)

Implications for the infrastructure: Only the following status changed should be allowed: development → production and production → deprecated.

Status comment header (optional)

/CMD_ComponentSpec/Header/StatusComment

This optional element offers a place for the owner of the component to store information regarding the reason for deprecation or succession.

Successor header (optional)

/CMD_ComponentSpec/Header/Successor

This optional element points to the direct successor of the component. It indicates a succession relation and is only meaningful if the values of /CMD_ComponentSpec/Header/Status is deprecated. The value should be the full URI of the successor component. Only a single successor can be specified. The successor relation thus is a 1:1 relation.

Implications for the infrastructure: Succession is a transitive relation. The status of the final component in a successor chain should be production. Users should be warned when deprecating a component without specifying a successor component.

Derived from header (optional)

/CMD_ComponentSpec/Header/DerivedFrom

This optional element points to the component from which the specified component was derived. This information gets stored at the moment a component is created (i.e. generally this will happen automatically in the Component Registry). No structural inheritance or even similarity is implied. This element is intended to be used for the construction of 'derivation trees' for metadata modelers to navigate, e.g. inside the Component Registry. Only a single component can be specified here.

Note that the derived-from relation is not the inverse of the successor relation.

Centre impact

The expected impact will be relatively small as implementation wise all required changes are in the 'metadata' of the component specification and can actually be ignored. The only software component that is affected in a large degree is the Component Registry, which will be responsible for gathering, storing and representing the status and versioning information.

Tools: Component Registry

Status can be mapped to views, see ComponentVersioning. Besides that, the Component Registry will have to provide inputs for the user to manage the status of their own components. Part of the information gathering can be automated (DerivedFrom).

Information about impact of status change should be readily available to the owner, especially when deprecating. For example, the number of components in which the component-to-be-deprecated is included.

Status filtering

The browsing mode of the Component Registry, as well as the REST service through which the list of components and profiles can be requested, needs to be filterable. Currently, there is a switch between private and public components. With the proposed set of status values, this needs to be revisited. A suitable paradigm seems to consist of two main views: components visible by the logged in user and components visible to the public - there may be an overlap. Then the following filter toggles are available for the latter view, both OFF by default:

• development components marked for review
• deprecated components

Usage warnings

The default filter settings described above are the first and most important step in discouraging users to use non-production components. In addition, warnings can be given at two stages:

• visually but non-obtrusive in the listing (e.g. warning icons on rows that represents the deprecated or development components)
• through a modal dialogue when a user actively decides to include such a component (through 'edit as new' or by dragging it into a component being edited). Should be checked recursively, i.e. also give warning whenever the the selected component contains deprecated components.
• visually in single component view (e.g. warning icon (same as in listing) beside any deprecated sub-components in the lower pane of CR)

Automatic 'derived from'

When a component is created by selecting an existing component and clicking 'edit as new', the DerivedFrom header value should automatically be populated with the ID of the original component. However it should be possible for the user to remove this or put another ID in place (but this should clearly be an advanced option!).

Owner/user notifications

The following automatic notifications would be nice to have:

• Notification of component owners when a component referenced (either directly or indirectly) in one of their components becomes deprecated
• Notification of component owners when a derivative of one of their components is created
• Notification of component subscribers when any of the subscribed components become deprecated

RSS or e-mail could be used as a means of notification/subscription.

Tools: metadata editing

Deprecated versions of profiles should not be advertised to the user (e.g. should not appear in the profiles list in Arbil) but there should be no restriction in using them (i.e. the can still be added by their schema URL) and existing metadata will be untouched. The metadata editor however could indicate the deprecated status of used profiles, including (non-obtrusive) notifications about the status of the profile the metadata being worked on is based on. For example, the icon shown for the metadata documents could reflect the state of the profile. Similarly, the list of active profiles in Arbil could mark those profiles in the list that are marked as deprecated. In case a successor is available, the editor should notify the user about that as well and ideally offer to load that profile instead (or in addition). Automatic upgrading to the new version should not happen, but assisting the user in attempting a migration could be considered (here the user should be warned about potential implications on the semantics of the existing values as a result!).

Tools: search & catalogue tools

Search tools that explicitly distinguish between profiles in the user interface could apply clustering with respect to different versions of the same profile (traversing the succession chain). In those cases where profile names are shown, it could make sense to show the status of the profile (e.g. under development, deprecated) if this is considered to be of interest to the user.

Implementation examples

Component specification:

<CMD_ComponentSpec isProfile="true">
   <Header>
      <ID>clarin.eu:cr1:p_1289827960126</ID>
      <Name>LrtInventoryResource</Name>
      <Description>Resources as stored before in the CLARIN LRT inventory</Description>
      <Status>deprecated</Status>
      <StatusComment>The following fields were missing: actor age, content language</StatusComment>
      <Successor>http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/profiles/clarin.eu:cr1:p_3989827960127</Successor>
      <DerivedFrom>http://catalog.clarin.eu/ds/ComponentRegistry/rest/registry/profiles/clarin.eu:cr1:p_1227960126456</DerivedFrom>
   </Header>

Tickets

Tickets in the CMDI 1.2 milestone with the keyword componentversioning:

Ticket	Summary	Owner	Component	Priority	Status
#146	Add a status field (development, production, deprecated)	Dieter Van Uytvanck	ComponentSchema	major	closed

Discussion

Oddrun: I agree with the proposed solution as far as it goes, and also like that both successor and derived-from are to be introduced. Some questions/issues:

• Only owners may declare a component deprecated, does that mean that only owners may create a successor/new version? Should the owner be obligated to create a successor to a deprecated component? This makes sense at least if the deprecated component is used in other components/profiles.
  • Menzo: Anyone (including the owner) can create a copy of a component, so at the moment of deprecation or later the owner can pick one of the copies (or even copies of the copies) of them as the successor. But he may also choose to select no successor in which case the component is end-of-life (for him). In problematic cases it will always be possible for the CR admin/content manager to pick a successor.
• Propagation of status change: In the proposed solution, steps are taken to prevent inadvertent use of deprecated components (remove them from public view). However, CMDI is a recursive universe, and any component may occur in other components and/or profiles. Consider the following figure:

When C2 is deprecated and a new version is available, this information should in some way be propagated to all its containing components and profiles. If not, both C1 and P1 will remain available in public. Any users selecting P1 (or another profile containing C2) for their metadata will then also choose the deprecated C2, without knowing that it is so. I really think that when a component is "replaced" by a new version, any "new" use of the old version should be strongly discouraged.

Menzo: This cascade could easily lead to lot of 'suddenly' problematic components and profiles if a frequently used component is deprecated, e.g., deprecation of cmd-language would cause a large ripple. I think its better to asses the lifecycle status per component, i.e., reuse of C1 is still fine even when its reuses the deprecated component C2. This is due to the fact that the owner of C2 can have very specific reasons for deprecation, which are not shared by the owner of C1. These conflicts of interest shouldn't prevent existing nesting from working, hence no cascades. The CR does try to discourage new nesting of deprecated components, but even that should still be possible.

Oddrun: But in "conflict-of-interest" cases you mention, in which a component successor represents an adaptation in response to some particular needs of the owner, then the deprecated one should NOT be removed from public view. Just as the owner of the existing C1 might want to go on using the (deprecated) C2, the creator of a new component might prefer C2 instead of its successor C4 as sub-component. (In this case it would probably be better not to deprecate C2, in stead just generate a new C4 based on C2 using the derived-from relationship. However, this will be entirely up to the component owners, and can at most be guided by best practice descriptions). All in all, the most logical outcome of your reasoning is I think to keep all deprecated components selectable from the public list, - and, whenever a deprecated component is selected for inclusion in a profile, inform the user about the successor.

Information about status change to existing component users: It might be nice to have some kind of subscription service, by which metadata creators, when choosing profiles for their metadata, get the opportunity to subscribe to notification whenever the profile or any of its components change status.

Menzo: Would indeed be a nice CR function.

---

Oddrun: After having read other parts of this wiki, I begin to think we should distinguish between updates (to a component) that qualify for state change, and those which do not. For instance, adding a documentation file to a component corresponds to updating its metadata, not the component in itself. Thereby, it has no consequences for its instances.

Menzo: This not a CMDI 1.2 issue but could be a nice addition for the CR. Still its tricky to assess what changes have a semantic impact, e.g., the documentation file might state some restrictions that were not explicit before as they can't be expressed by CMDI, e.g., if element A has a value X then optional element B should have a value.

---

Summary of conclusions from WG e-mail discussions (2014-2-10):

• Model aspects (component specification header) are settled
• Only owner (and CR administrator) can change the lifecycle status of a component in the registry
  • But anyone can make a derivative of any component
• Lifecycle status should not propagate to referencing components (i.e. component X that includes component Y does not change status when component Y becomes deprecated)
• Modellers should be discouraged from using deprecated components but not disallowed. Default filtering should hide deprecated and development components in the registry
• Metadata creators should be discouraged from instantiating deprecated profiles by similar means

Discuss the topic in general below this point