Revision as of 15:03, 12 April 2010

Details of existing and proposed Extensions to the DAS 1.6 specification, once it has been released.

Existing Extensions

These extensions have undergone development and have solid implementations.

Interaction

The DASMI extension expands DAS to apply to molecular interactions. It is part of the DAS 1.53E specification (i.e. is an existing extension to the 1.53 specification). It is detailed here.

DAS writeback

June 10,2009

This is a working document and a proposal for an extension to the DAS 1.6 specification in order to support writeback capabilities. A writeback server should have, at least, the methods for the basic reading/writing operations, which in Databases is recognized as CRUD(Create, Read, Update and Delete). The reading component is already solved for the DAS protocol, even is possible to affirm that all the current commands in the specification are reading commands for different kind of information.

One or more Writeback servers can be associated to a coordinate system, however just one coordinate system can be handle for a writeback source, and in such a way be sure to identify the feature with the correct segment of the right specie.

DAS is partially following the concepts of RESTful services, and that is the reason why most of this proposal is inspired in the RESTful concept of Uniform Interface, which means that all the resources should be manipulated using a predefined set of operations. DAS use HTTP(see RFC2616) as its communication protocol, therefore the logic Uniform Interface for DAS is the use of the HTTP methods PUT, GET, POST and DELETE to access the 4 CRUD operations. Next, is explained the proposed details to use this methods with DAS, but first a description of the das writeback document.

DAS Writeback Document

The document used for all the methods is an XML-formatted "DASGFF" document. All the information of an annotation to create/edit could be supplied using this format, which implies that the implementation of this extension is dependent of the DAS version implemented. All the elements of this format are explained in the DAS 1.6 specification. An example of the Document is below, and as it will be explained later the same format could be used for the input or the output of one of the HTTP methods. Extra information can be required depending of the implementation as metadata of the operation. For those cases the element NOTE should be used in a notation KEY=VALUE. In the example below this notation is used to represent the OpenId of the user who sent or is sending this feature to the server.

<?xml version="1.0" standalone='no'?>
<!DOCTYPE DASGFF SYSTEM "http://www.biodas.org/dtd/dasgff.dtd">
<DASGFF>
 <GFF version="1.0" href="http://www.ebi.ac.uk/das-srv/uniprot/das/uniprot/features?segment=P05067">
   <SEGMENT id="P05067" start="1" stop="770" version="7dd43312cd29a262acdc0517230bc5ca">
      <FEATURE id="UNIPROTKB_P05067_KEYWORD_Disease" label="Disease mutation">
       <TYPE id="BS:01019" category="inferred by curator (ECO:0000001)">disease</TYPE>
       <METHOD id="UniProt">UniProt</METHOD>
       <START>10</START>
       <END>40</END>
       <SCORE>0.0</SCORE>
       <ORIENTATION>0</ORIENTATION>
       <PHASE>-</PHASE>
       <LINK href="http://www.uniprot.org/uniprot/P05067">http://www.uniprot.org/uniprot/P05067</LINK>
       <NOTE>Adding a new feature!</NOTE>
       <NOTE>USER=http://user.myopenid.com</NOTE>
     </FEATURE>
   </SEGMENT>
 </GFF>
</DASGFF>

HTTP METHODS (DAS writeback methods)

POST

The HTTP method POST should be used to create a feature in the server. The writeback document should be sent to the server in a POST variable called content . The server should create an identifier for this feature. This id should be the URI formed by the concatenation of the URL of the writeback server and some unique number for the server(a consecutive number). The response should be based in the HTTP headers of the DAS specification. (i.e. 200 OK, 500 Server error, etc.). The content of the response should be a GFFDAS format with the information that was created in the database therefore if everything is correct the response will be almost the same that the input document. The only difference will be the metainformation added for the server as notes, for example

<NOTE>USER=http://user.myopenid.com</NOTE>
<NOTE>VERSION=1</NOTE>
<NOTE>DATE=2009-06-10 18:11:30.672644</NOTE>

PUT

The behavior of the method PUT is very similar to the described for POST. However in this case the writeback document is the content itsel of the request(i.e. Is not embedded in any parameter). An important difference is the management of the id, which is defined as the value in the field id of the element FEATURE if this value is a valid URI, otherwise is the URI formed by the base URL in the field href concatenated with the value in the field id of the element FEATURE. For example for the same writeback document of above the id in the element FEATURE will be:

<FEATURE id="http://www.ebi.ac.uk/das-srv/uniprot/das/uniprot/features/UNIPROTKB_P05067_KEYWORD_Disease" label="Disease mutation">

The response should follow the same rules as the POST method.

DELETE

The DELETE method doesn't require a writeback document for the input, because in order to DELETE a feature is just necessary to identify it. The identification of a feature will be reconstructed by its id and the id of the segment where it will be deleted. The openid is also require for this transaction. The format of the HTTP connection should be:

DELETE [server]?featureid=[featureid]&user=[openid]&segmentid=[segmentid]

A successful response SHOULD be 200 (OK) if the response includes an entity describing the status, 202 (Accepted) if the action has not yet been enacted, or 204 (No Content) if the action has been enacted but the response does not include an entity. The content of the response should be in the same GFF format but the parameter label of the element FEATURE should be DELETED as in:

<FEATURE id="http://writeback/92" label="DELETED">

GET

All the commands of reading in the DAS specification should Apply here, however the deleted features should be also included here(in the same format as explained above) and is the client who decide how to used this information. There is an extra command that a writeback source should implement:

Historical Command

Retrieve all the versions that the writeback has for an specific feature.

Scope: Writeback servers.

Command: historical

Format:

PREFIX/das/DSN/historical?feature==FEATUREID

Description: This query returns all the versions for an specific feature embedded in its respective segment(s).

Arguments:

feature (required; one)

URL identifier of a particular feature in the writeback.

Here is an example of a valid request:

    http://www.writeback.com/das/writeback/historical?feature=http://writeback/9

Response:

The document returned from the features request is an XML-formatted "DASGFF" document.

Format:

<?xml version="1.0" standalone="no"?>
<!DOCTYPE DASGFF SYSTEM "http://www.biodas.org/dtd/dasgff.dtd">
<DASGFF>
 <GFF version="1.0" href="http://localhost:8080/MyDas/das/writeback/historical?feature=http://writeback/9">
   <SEGMENT id="P05067" start="1" stop="770" version="7dd43312cd29a262acdc0517230bc5ca">
     <FEATURE id="http://writeback/9" label="Disease mutation">
       <TYPE id="BS:01019" category="inferred by curator (ECO:0000001)">disease</TYPE>
       <METHOD id="1">UniProt</METHOD>
       <START>143</START>
       <END>189</END>
       <SCORE>0.0</SCORE>
       <ORIENTATION>0</ORIENTATION>
       <PHASE>-</PHASE>
       <NOTE>testing note</NOTE>
       <NOTE>USER=http://user.myopenid.com</NOTE>
       <NOTE>VERSION=1</NOTE>
       <NOTE>DATE=2009-05-25 14:22:39.705735</NOTE>
       <LINK href="http://www.uniprot.org/uniprot/P05067">http://www.uniprot.org/uniprot/P05067</LINK>
     </FEATURE>
     <FEATURE id="http://writeback/9" label="DELETED">
       <TYPE id="" category="" />
       <METHOD />
       <START>0</START>
       <END>0</END>
       <SCORE>0.0</SCORE>
       <ORIENTATION>0</ORIENTATION>
       <PHASE>-</PHASE>
       <NOTE>USER=http://user.myopenid.com</NOTE>
       <NOTE>VERSION=2</NOTE>
       <NOTE>DATE=2009-06-10 17:58:11.83588</NOTE>
     </FEATURE>
   </SEGMENT>
 </GFF>
</DASGFF>

User Authentication

The control of who is authorized or not to made modification in the writeback is an source implementation issue, however the authentication of the user should be done using OpenId as seen in the examples above the openid user should be sent through the NOTE element:

<NOTE>USER=http://user.myopenid.com</NOTE>

Proposed Extensions

These extensions are merely proposals for future modifications, and do not yet have implementations.

DAS search

A mechanism for programmatic search of content within annotation servers.

Support for alternative content formats

Content negotiation, either via request parameters or HTTP auto-negotiation. This would allow support for response formats other than DAS XML. Examples might include JSON, XHTML, etc.

Rationale

DAS XML is expressive but can be very verbose. This is particularly problematic for the features query, which depending on server and query can result in extremely large responses. It is therefore desirable to allow more efficient alternative content formats to be returned within the general DAS query framework, particularly for feature queries.

Another advantage is simplicity. If we allow a DAS server to send back other formats, then it can take advantage of libraries such as Picard or tools like samtools to run region queries on indexed flat files and return the data without having to parse and process them. All the server would have to do is decode the request URL and then use Picard or samtools to grab the data from the flat file and return it to the client.

Proposed Implementation

At the DAS workshop the following was discussed as an alternative to the initial proposal below.

Can query a datasource for a list of the formats it supports for example ../das/hg18/format should result in something like the following:

<DASFORMAT>
 <COMMAND name="das1:features">
   <FORMAT name="das-JSON">
..if no types specified here then all types for this source have this format for this command
     <TYPE id="gene"/>
     <TYPE id="exon"/>
   </FORMAT>
   <FORMAT name="das-GoogleProtocolBuffers">
   ..if no types specified here then all types for this source have this format for this command
     <TYPE id="gene"/>
     <TYPE id="exon"/>
   </FORMAT>
 </COMMAND>
<COMMAND name="das1:entry_points">
   <FORMAT name="das-JSON">
   </FORMAT>
 </COMMAND>
</DASFORMAT>

Initial proposal

Addition of an optional zero-or-more <FORMAT> element as a child of <TYPE> element in types command response, with required attributes "name" (arbitrary character string that uniquely identifies this format within the server) and "mimetype" (must follow standard mimetype identifier rules). Note that this strategy also allows for multiple formats with the same mimetype.

Addition of an optional "format" query parameter to features query. Format parameter SHOULD be a format name that the server recognizes. If server can return ALL features that satisfy the feature query in the specified format, it should do so, and set the response Content-Type header accordingly. If the server does not recognize the requested format, or cannot return in that format all of the features that satisfy the features query, it should return an HTTP error message (with X-DAS-Status also set?).

Both of these additions are optional, therefore these changes will not affect servers or clients that do not support alternative content formats.

Use of URIs for DAS identifiers

Often, the use of unique IDs within DAS is poorly executed. Formally adopting URIs as identifiers (including specifying how URIs are built from URI references within DAS XML documents) would allow cross referencing between sequences and annotations within and outside DAS.

Entry Points for annotation servers

January 28, 2010

Rationale

In DAS version 1.6, the entry_points command is required for reference servers and optional for annotation servers.

However, it is difficult for annotation servers to support this command because the start/stop attributes of the SEGMENT element are mandatory. In contrast to reference servers, annotation servers very rarely know the length of each entry point and therefore cannot satisfy this requirement. This is a pity because it'd be useful for clients if annotation servers were able to provide a list of IDs for the segments it has annotated. This would be very easy to implement because every server always knows the IDs of the segments it has annotated, and needs this information in order to support the unknown-segment capability.

Required Change

Allowing annotation servers to easily list their entry points would require amending the specification to say something like:

• reference servers must always list all possible segments with their start/stop positions
• annotation servers implementing entry_points must list only the segments they have annotated, and start/stop are optional