{{{ #!html

Contents

}}} [[PageOutline(1-3, , inline)]] {{{ #!comment Obviously, your page starts below this block }}} = XSD Schema = == Preamble == The xsd schema is designed according to the following paradigm: -- There are 7 sorts of resources in DASISH: {{{CachedRepresentation}}}, {{{Source}}}, {{{User}}}, {{{Annotation}}}, {{{Notebook}}}, {{{Lists of Permissions}}}, {{{Lists of Versions}}}. -- There are 6 xsd-types corresponding to the serialisations of all the types of resources above, except {{{CachedRepresentation}}}. There is no an xsd-schema type corresponding to {{{Cached representation}}} because a cached representation is a "pure" resource like an image or a text file that does not contain any meta-information about itself. The metadata of a cached presentation are defined via an instance of {{{CachedRepresentationInfo}}}. -- Each of these 6 types has an obligatory attribute "URI" which contains DASISH identifier pointing to the location of the resource on the DASISH server. -- There are corresponding lists-of-reference types: {{{CachedRepresentations}}}, {{{Sources}}}, {{{Users}}}, {{{Annotations}}}, {{{Notebooks}}}. Their names are just plural English forms of the corresponding types. -- There are corresponding resource-info types: {{{CachedRepresentationInfo}}}, {{{SourceInfo}}}, {{{UserInfo}}}, {{{AnnotationInfo}}}, {{{NotebookInfo}}}. They contain reference to the corresponding resource plus the most important information about the resource. -- There are corresponding list-of-resource-info types: {{{SourceInfos}}}, {{{UserInfos}}}, {{{AnnotationInfos}}}, {{{NotebookInfos}}}. There is a number of auxiliary types as well. A commonly-used one is ResourceREF which contains the attribute "ref" of type {{{xs:anyURI}}}. It allows to declare elements-references and avoid mixing them with elements-resources. === Handling new (not yet in the DB) sources === Adding annotation with the target sources which are not yet in the DB needs special treatment. It becomes clear when the POST body for a new annotation must be serialized. Two approaches seem to be plausible. We will follow the FIRST option. 1) A "strongly-typed" schema. An annotation contains a list of elements-"targets". Each of them can be either a source element or a new-source element. It is implemented using xs:choice construct for elements. A source and a new-source element differs by one attribute: a source has obligatory "ref" attribute, and a new source has an obligatory "xml:id" attribute. See [source:DASISH/t5.6/schema/trunk/annotator-schema/src/main/resources/DASISH-schema.xsd DASISH-schema] 2) A "weakly-typed" schema. An annotation contain a list of elements-"targets" of the same type that contains two non-obligatory attributes: "ref" and "xml:id". The type-checking "''at least one of the attributes is present and they are mutually exclusive''" may be left for later to schematron or so. See [source:DASISH/t5.6/docs/XMLandXSD/DASISH-schema-alternative.xsd DASISH-alternative-xsd]. The link to the second, "weakly-typed", version of the XSD-schema is left for the reference, however this version is not maintained any more. = Scenario XML's validated vs the given schema = See [[DASISH/Scenario]] == Responding GET api/user/uid == {{{#!xml }}} == Retrieving annotations == === Responding GET api/annotations?link="http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia"&access=read === The root element below is of type {{{AnnotationInfos}}} (the list of {{{AnnotationInfo}}}). {{{#!xml My client is not in a hurry Nativity Facade Nativity Facade (old site) }}} === Responding GET api/annotations/AIDzzz (example of resolvable target sources) === {{{#!xml Nativity Facade different http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia#Nativity_Fa.C3.A7ade 20.04.2013 http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia#Passion_Fa.C3.A7ade 20.04.2013 }}} === Responding GET api/annotations/AIDzyy (example usage for unresolvable target sources 1) === The respond for an annotation with unresolved target sources and the respond for an annotation with resolved target sources (see above) are both instances of the same schema element. However, the annotation refers to an obsolete version of the page. Next, having the target source references, the client will ask for the source versions saved in the DB. The last step: having the info about the version under consideration, the client asks for cached representations of the version. {{{#!xml ?xml version="1.0" encoding="UTF-8"?> Nativity Facade (old page) different http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia#Nativity_Fa.C3.A7ade 2010-01-29T23:59:59 http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia#Passion_Fa.C3.A7ade 2010-01-29T23:59:59 }}} === Responding GET api/sources/SIDbbb (unresolvable target sources 2) === {{{#!xml }}} === Responding GET api/sources/SIDbbb/versions (unresolvable target sources 3) === {{{#!xml 2010-01-29T23:59:59 20.04.2013 }}} === Responding GET api/sources/SIDbbb/cached/CIDtttt/metadata (unresolvable target sources 4) === {{{#!xml }}} == Making a new annotation == === Request body for POST api/annotations === {{{#!xml Comapring English and Catalan Wiki History http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia#History 20.04.2012 http://ca.wikipedia.org/wiki/Temple_Expiatori_de_la_Sagrada_Fam%C3%ADlia#Hist.C3.B2tia 20.04.2013 }}} === Request body for POST api/annotations: another example === The serialization of the POST body for another example (UGOT): {{{#!xml Douglas Adams - Wikipedia, the free encyclopedia Adams was born 1952-03-11 http://en.wikipedia.org/wiki/Douglas_adams#xpointer(start-point(string-range(//div[@id="mw-content-text"]/table[1]/tbody[1]/tr[3]/td[1]/text()[1],'',12))/range-to(string-range(//div[@id="mw-content-text"]/table[1]/tbody[1]/tr[3]/td[1]/text()[1],'',25))) 2013-04-26T11:23:26.000Z }}} === Response body (envelope) for POST api/annotations === The temporary id is replaced with the permanent reference. However, no cahced representation is found for the catalan web-page. Therefore, in the action part of the envelope there is an action CREATE_CACHED_REPRESENTATION for the object which is the source for catalan web-page. {{{#!xml Comapring English and Catalan Wiki History http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia#History 20.04.2012 http://ca.wikipedia.org/wiki/Temple_Expiatori_de_la_Sagrada_Fam%C3%ADlia#Hist.C3.B2tia 20.04.2013 }}} The client sends metadata cached representation in the POST body, and a cached representation itself. An example of serialized metadata for a cached representation has been considered above, so we do not give it here. == Editing annotation body == === Request: an updated body === {{{#!xml History in English and Catalan }}} === Enveloped respond: new (updated) annotation and a list of actions === The list of actions is empty because there are cached representations for all the target sources. {{{#!xml Comapring English and Catalan Wiki History in English and Catalan http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia#History 20.04.2012 http://ca.wikipedia.org/wiki/Temple_Expiatori_de_la_Sagrada_Fam%C3%ADlia#Hist.C3.B2tia 20.04.2013 }}} == Managing permission lists of users == === GET api/annotations/AIDzyy/permissions === {{{#!xml }}} === GET api/users/info?email="alecor@mpi.nl" === {{{#!xml }}} === PUT api/annotations/AIDzyy/permissions=== Swapping rights of the users xyxy and xxzz: xyxy one becomes a reader, and xxzz becomes a writer. Updating w.r.t. the respond {{{GET api/annotations/AIDzyy/permissions}}} above. Request body: {{{#!xml }}} Respond body: is an envelope containing this list and no actions, since all the users are presented in the DB. === PUT api/annotations/AIDzyy/permissions/UIDagc === {{{#!xml writer }}} == Managing Notebooks == === GET api/notebooks === {{{#!xml Gaudi Douglas Adams }}} === GET api/notebooks/NIDxyxy === {{{#!xml Gaudi }}} === GET api/notebooks/NIDxyxy/annotations/ === Respond is a list of annotation info, is similar to the respond on {{{GET api/annotations?link="http://en.wikipedia.org/wiki/Sagrada_Fam%C3%ADlia"&access=read}}}. = Issues with the schema = == Possible namespace pollution: Ticket 348 to be discussed with Peter == See [https://trac.clarin.eu/ticket/348] Peter: "It looks like there might be some namespace pollution or some other anomaly that causes the jaxb a uto generated classes to omit the getter for notebooks from the ObjectFactory. This is an issue when a jaxb root node is required, such as in the rest interface. A work around has been added which makes it clear where the issue is and why the ObjectFactory is required, but this needs to be replaced when the schema is updated."[[BR]] [[BR]] ''Stephanie:'' If indeed, the issue lies with the plural forms of complexType names (used for lists of resources) in the schema, we think that, principally, the plural forms can - and should - be replaced more or less immediately, e.g. by adding the suffix "{{{List}}}" or "{{{Collection}}}" instead of plural "s". For instance, the serialized xml output could then match the following snippet {{{ }}} By the way, speaking of diverse "JAXB plural issues", there were some hints on the web regarding (inline and external) Java binding customization for JAXB, - it might be that this would also be a viable solution approach. Maybe this thread is worth checking as well: http://stackoverflow.com/questions/11943487/please-advise-the-best-pattern-for-serialising-jaxb-lists? As regards all future schema updates, please make sure to remember to update all available scenario xml documents on trac.clarin.eu accordingly, and preferably, with as little delay as possible. It might also be a good idea to validate all of these documents against the revised schema with a reliable XML parser like e.g. the Xerces-J XML parser. Also, if you like, you can check by validating some of our current "real-life" mock xml documents that we have been using recently for client development ([source:DASISH/t5.6/client/trunk/chrome/markingcollection/content/markingcollection/annotator-service/test/mockjax/mocks]). === Resolution === The schema is to be fixed. The current hypothesis is that JAXB get confused by plural forms used in the schema in type names for lists of resources, e.g. Annotation --> Annotations, Notebook --> Notebooks etc. Replace plural forms with other wording and check if it fixes the problem. == New-Or-Existing-Source-Info JAXB-generated class == This class corresponds to the choice sub-schema within the schema-type for annotation, which tells if a target source is new or old. This means that the client provides the server with this information. It is ok, but it makes code a bit hairy. Can we remove the choice and let the backend todecide if the source is new or old, simply by looking through all the sources to find the one with given external-id/uri. If it is not found then the source is new. [[BR]][[BR]] ''Stephanie:'' Both Olof and I are somewhat unsure whether it is really necessary to differentiate between new and existing sources when sending POST, PUT, DELETE and GET requests from within the client, especially if the backend implementation is going to be changed to simply check whether the given URI already exists in the DB or not. Could you please rethink whether we actually need to keep this concept in the schema document, DASISH-schema.xsd? This decision also has consequences for how the behavior of the client needs to be tuned (cf. [source:DASISH/t5.6/client/trunk/chrome/markingcollection/content/markingcollection/annotator-service/test/mockjax/mocks POST/GET] mock docs named above). === Resolution === The schema stays intact. For the backend-code: the server should check if the source is indeed new, see https://trac.clarin.eu/ticket/362 for more detail. == External_id (Data Base) vs URI (schema) vs UUID-based class (Java code) == For the time being I treat {{{extrnal_id}}} and the URI as "the same": URI is external_id. Both are strings. In the Java-code there are classes {{{{CachedIdentifier, VersionIdentifier, SourceIdentifier, AnnotationIdentifier, NotebookIdentifier, UserIdentifier}}} encapsulating UUI. Any such class has a string field "identifier" (corresponding to external_id) plus hash, plus internal constants for hash. === Resolution === The schema stays intact. For the back-end code: the URI must be of the form "http(s)://externalID. See https://trac.clarin.eu/ticket/363 for more detail. Moreover, all {{{{CachedIdentifier, VersionIdentifier, SourceIdentifier, AnnotationIdentifier, NotebookIdentifier, UserIdentifier}}} must be removed. We will be using just UUID type for all identifiers of any sort of resource. E.g. UserIdentifier userIdentifier = New UserIdentifier("00000000-0000-0000-0000-000000000003") will be replaced with UUID userIdentifer = UUID.fromString("00000000-0000-0000-0000-000000000003"). It is the ticket https://trac.clarin.eu/ticket/371 == Body: must be some serialization/deserialization mechanism == -- "body" in the DB is just a text -- "body" in JAXB-generated class is a list of objects For now, I use simple "serialize" and "deserialize" Helpers' procedures which should be replaced by some proper marshalling-demarshalling. For simple serialization I treat the first element of the list of objects above as a text whcich corresponds to the DB column "body_xml". === Resolution === The schema stays intact. For the back-end code: still needs to be discussed "how we are going to parse annotation bodies". See https://trac.clarin.eu/ticket/364 for more detail. [[BR]][[BR]] ''Stephanie:'' It might be preferable to abandon the helper methods in favor of using JAXB's marshal() and unmarshal() methods together with a(n) {{{Marshaller}}} / {{{Unmarshaller}}} object. Can JAXB's {{{ObjectFactory}}} methods be of any use as an alternative way to access XML data? Furthermore, you might want to have a look at Javax.xml.bind.JAXBIntrospector.getValue(Object jaxbElement). Please see: http://www.tutorialspoint.com/java/xml/; http://www.oracle.com/technetwork/articles/javase/index-140168.html; http://docs.oracle.com/javase/tutorial/jaxb/intro/index.html; http://jaxb.java.net/tutorial/ - in case you haven't come across these web resources before. == Source == Misprint in timeStamp: timeSatmp. === Resolution === The schema must be fixed. See https://trac.clarin.eu/ticket/365 == Cached Representation Info == Missing in the schema: the attribute/elememt "where_is_the_file" which actually points to the location where the file can be download. It is necessary to fulfill GET api/sources//cached//content === Resolution === The schema stays intact. For the back-end (and the database!): the cached representation for now should be stored in the database as a BLOB. See: http://dba.stackexchange.com/questions/803/blobs-or-references-in-postgresql/815#815. Also, see https://trac.clarin.eu/ticket/366 == Version == MISSING in the schema: attribute URI (corresponding to the external_id in the DB) is absent. Therefore it does not appear in the JAXB-generated class "Version" and the java class has one attribute less than the DB table "version" {{{ CREATE TABLE version ( version_id SERIAL UNIQUE NOT NULL, external_id UUID UNIQUE NOT NULL, version text, ); }}} For now I'm using attrribute "version:String" now to keep "external_id/URI" in the java class "Version". === Resolution === Fix the schema: URI must be added to the version-type. The ticket: https://trac.clarin.eu/ticket/367 == LISTS of Resources, like "PermissionS" and "CachedRepresentationS", and version-siblingS connected to a particular source cannot be standalone tables in the relational DB == According to the schema: a list of Cached representations is declared as a standalone resource of type "CachedRepresentations". Every version referres to its own list of cached representations. Every such list has it own ID. According to the Rel. database: it looks a bit strange to have such lists. Instead, i have made a common joint table (verson_id, cached_representation_id). A pair (a, b) is listed in this table iff the version with the internal id "a" has cahced representation with intrenal id "b". === Resolution === Fix the schema: all lists of resources are artifacts, they do not have URI's. Ticket: https://trac.clarin.eu/ticket/368