This is a working document and a proposal for a reworked DAS specification which hopes to clarify the DAS spec based on how DAS is being used in the community today and to include commands from the <a href="http://www.dasregistry.org/spec_1.53E.jsp">1.53E spec</a> and some of the 2.0 spec. Also the document has been adjusted to reflect changes in the use of DAS away from a solely genome centric protocol to a more open one encompassing other reference/coordinate systems such as alignments and protein structures. The spec also includes references to the DAS Registry which is essential for implementing an SOA architecture. Note: this is a technical document but should be readable and understandable by people without a deep understanding of broader technical issues and other system architectures.

==Overview of the System==

This section provides a high-level view of the system architecture.

The DAS system consists of the a reference server, one or more annotation servers and the das registry.

====The Reference Server====

Central to the DAS system is the idea of reference objects that can be served from a reference server.

These are biological data objects with stable identifiers which are targets for annotation. In the original DAS protocol, the reference objects are always biological sequences: chromosomes, scaffold sequences from genome assemblies or protein sequences. When a DAS client starts, its first action is to connect to an appropriate reference server and retrieve the reference sequence. Once a reference object has been loaded, the DAS client will contact one or more annotation servers to obtain the data provided for the reference objects. Typical annotations might include sets of predicted exons on a genome sequence, or matches to a protein domain model on a protein sequence. It is the client's responsibility to collect all relevant annotations and display them in a user-friendly manner.

<p> A coordinate system describes the data that is made available by a DAS source/reference server. This information is important for the DAS clients, to deal with data correctly, as they often can accept data served in multiple coordinate systems.

The coordinate systems are described by four fields: Authority, (assembly) Version, Type, and Organism. The assembly version is important for genome assemblies, but not really applicable for other datasets like UniProt sequences, therefore this field is optional.

the reference objects.Annotations have an ID that is unique to

"intron", "CDS" and "splice3."(We encourage the use of the http://www.sequenceontology.org/ Sequence Ontology IDs to give uniformity to DAS sources for example CDS is SO:0000316). The annotation method (<font color="red">what are we doing with annotion method if catagory is handling this??</font><b>method</b> is intended to describe how the annotated feature was discovered, and may

To look up  ontologies you could go here: "http://www.ebi.ac.uk/ontology-lookup/"

<b>category</b> is a broad functional category that can be used to

filter, group and sort annotations.  "Homology", "variation" and

"transcribed" are all valid categories.  The existence of these

categories allows researchers to add new annotation types if the

value.  (we also encourage the use of ECO numbers to represent the method of annotation e.g.ECO:0000032 "inferred from curated blast match to nucleic acid).

<TYPE id="SO:0000417" category="inferred from reviewed computational analysis (ECO:0000053)">polypeptide_domain</TYPE>

 

 

 

 

  <li>Given a reference sequence id, it can return the raw sequence.

  <li><font color="red">Deprecate this? Given a reference sequence id, it can return annotations of the

      category "component". Component annotations describe how the

      sequence is assembled from smaller parts into large parts from the

  <li>Given a reference sequence id, it can return annotations of the

      describe the assembly of the sequence from the bottom up.

Although the servers are conceptually divided between reference

between them. A single server can provide both reference sequence

information and annotation information. The main functional

difference is that the reference sequence server is required to serve

the sequence map and the raw DNA, while annotation servers have no

such requirement <font color="red">how to generalise this? What does not serve sequence info?</font>.

central DAS registry which implements this protocol and fulfils the following roles:

1. It allows discovery of available DAS sources via a web page, or as machine-readable XML which can be used directly by DAS client programs.

2. It automatically validates registered DAS sources to ensure that they return well formed DAS XML.

3. It periodically tests DAS sources and notifies their administrators if they are unavailable.

4. It can group the registered DAS sources according to the coordinate systems of their data.

The distributed annotation system (DAS) relies on there being a common

"co-ordinate system" on which to base annotations.  The co-ordinate system consists of a set of "entry points", and

the lengths of each entry point <font color="red">but not all in the case of alignments??</font>. The identity of an entry point will

vary from co-ordinate system to co-ordinate system.  For some projects, entry points

correspond to entire chromosomes.  For others, entry points may be a

<font color="red">remove this section on subsequence?</font>It is possible for each entry point to have substructure, basically

subsequences within the entry point.<br/>

The DAS is Web-based. Clients query the reference and annotation servers by sending a formatted URL request to the server. This request must follow the conventions of the HTTP/1.0 protocol (see [ftp://ftp.isi.edu/in-notes/rfc2616.txt RFC2616]. Servers process the request and return a response in the form of a formatted XML document (see [http://www.w3.org/XML/ W3C Extensible Markup Language]).

All DAS requests take the form of a URL. Each URL has a site-specific prefix, followed by a standardized path and query string. The standardized path begins with the string '''/das'''. This is followed by URL components containing the data source name and a command. For example:

 

  http://www.wormbase.org/db/das/elegans/features?segment=CHROMOSOME_I:1000,2000

  ^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^ ^^^^^^^ ^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    site-specific prefix   das  data  command  arguments

In this case, the site-specific prefix is '''http://www.wormbase.org/db'''. The request begins with the standardized path '''/das''', and the data source, in this case '''/elegans'''. This is followed by the command '''/features''', which requests a list of features, and a query string providing named arguments to the '''/features''' command.

The data source component allows a single server to provide information on several genomes.

'''<font color="red">SOAP has not been widely adopted for das so should we delete this?</font>''' The request may be replaced with a SOAP-style XML-encapsulated document in future versions of this specification.

The response from the server to the client consists of a standard HTTP header with DAS status information within that header followed optionally by an XML file that contains the answer to the query. The DAS status portion of the header consists of two lines. The first is X-DAS-Version and gives the current protocol version number, currently DAS/1.0. The second line is X-DAS-Status and contains a three digit status code which indicates the outcome of the request.

  X-DAS-Version: DAS/1.5

===New in version 1.5===

The X-Das-Capabilities header provides an extensible list of the capabilities that the server provides. This can be used by those writing experimental extensions to DAS to flag clients that those extensions are available. Capabilities have the form ''CapabilityName/Version'' and are separated by semicolon, space, as in "capabilityA/1.0; capabilityB/1.4; capabilityC/1.0". The following standard capabilities are present in the DAS/1.5 protocol:

| <font color="red">Deprecate these: The server supports the basic ''dsn'' request.</font>

| <font color="red">The server supports the basic ''dna'' request.</font>

! align="LEFT" | stylesheet/1.0

| The ''features'' request will accept the CGI parameter "feature_id", enabling the server to look up segment(s) based on the ID of a feature.

| The ''features'' request will accept the CGI parameter "group_id", enabling the server to look up segment(s) based on the ID of a group.

| The ''features'' request will return components of the indicated segment when a category type of "component" is requested.

| The ''features'' request will return supercomponents of the indicated segment when a category type of "supercomponent" is requested.

| The server supports the new ''sequence'' request.

==Reference sequence IDs==

 

A reference ID can contain any set of printable characters (including the space character), but not the colon character (":"), which is reserved for separating reference IDs from sequence ranges (see below). The newline, tab and carriage return characters are also reserved for future use.

A data source that uses the colon character for its internal IDs must map this character to another one on the way out and on the way in. For example:

This section lists the queries recognized by reference and annotation servers. Each of these queries begins with some site-specific prefix, denoted here as ''PREFIX''. The other meta-variable used in these examples is ''DSN'', which is a symbolic data source. (As seen the the [#request  above example.]) Data sources are standardized across DAS servers in such a way that a data source name has a one-to-one correspondence with a reference sequence.

'''Command:''' ''Sources''

<li>The <a href="help_coordsys.jsp">coordinate system</a>(namespace) of the provided data</li>

<li>different properties that allow to describe a server closer</li>

<b>Scope:</b> all DAS servers

 

<b>Command:</b> <i>sources</i>

 

 

 

 

 

 

 

 

 

The response to the <i>sources</i> command is the "DASSSOURCE" XML-formatted document:

<td>an XSL stylesheet that e.g. allows a browser to nicely display the XML response </td>

<td>in principle this would allow hosting several versions of a DAS sources (with unque uris)  

provide only the server with the latest data. the created attribute provides the date on which a DAS server has been set up initially.

<td>The description of the namespace of a DAS source. <br/>

<b>uri</b> - the unique URI for a DAS source. For a DAS registration server these

should be resolvable and allow to access more information about this.  

e.g. <a href="http://www.dasregistry.org/dasregistry/coordsys/CS_DS6">http://www.dasregistry.org/dasregistry/coordsys/CS_DS6</a>

for the UniProt,Protein Sequence coordinate system. <br/>

<li> <a href="http://www.dasregistry.org/das1/sources">http://www.dasregistry.org/das1/sources</a>

- the listing of DAS/1 sources at the DAS registration server</li>

<li> <a href="http://www.ensembl.org/das/sources">http://www.ensembl.org/das/sources</a>

<li> <a href="http://vega.sanger.ac.uk/das/sources">http://vega.sanger.ac.uk/das/sources</a>

===Retrieve the List of Entry Points for a Data Source (entry_points cmd)===

: There is a single <ENTRY_POINTS> tag. It has a version number (required) in the form "N.NN". Whenever the DNA of the entry point changes, the version number should change as well. The '''href''' (required) attribute echoes the URL query that was used to fetch the current document.

===<font color="red">Retrieve the DNA Associated with a Subsequence has been deprecated as the sequence cmd below is more commonly used.</font>===

===Retrieve the Sequence Associated with a Subsequence (sequence cmd)===

<blockquote><pre>

<i>PREFIX</i>/das/<i>DSN</i>/sequence?segment=<i>RANGE</i>[;segment=<i>RANGE</i>...]

</pre></blockquote>

 

     http://www.wormbase.org/db/das/elegans/sequence?

         segment=BUM;segment=HUM_HGA;segment=CE_HOC2:1,200

====The Sequence Response====

 

 

   &lt;SEQUENCE id="id" start="start" stop="stop"

   <dd>There is a single &lt;SEQUENCES&gt; tag per requested segment. It has the  

attributes <b>id</b>, which indicates the reference ID for this sequence,

      <b>start</b> and <b>stop</b>, which indicate the position of

      this segment within the reference sequence, <font color="red">Deprecate this as not consistent with registry and coordinate system?<b>moltype</b>,

      which indicates the molecular type of the sequence, and <b>version</b>,

      which provides the sequence map version number.  All five

      attributes are required.

 

       <p>

 

===Retrieve the Types Available for a Segment (types cmd)===

: One or more type IDs to be used for filtering annotations on the type field. If multiple type names are provided, the resulting list of features will be the logical OR of the list. For compatibility with versions 0.997 and earlier of this protocol, servers are allowed to treat the type ID as a regular expression, but this feature is '''deprecated''' and should not be used.

       <TYPE id="''id1''" method="''method''" category="''category''">''Type Count 1''</TYPE>

       <TYPE id="''id2''" method="''method''" category="''category''">''Type Count 2''</TYPE>

: The <SEGMENT> tag provides information on the reference segment. The '''id''', '''start''' and '''stop''' attributes indicate the coordinate system of the segment, and are required if the segment corresponds to a defined region of the genome, optional if the list of types corresponds to the entire database. The '''version''' attribute (required) indicates the current version of the sequence map. The optional '''label''' attribute supplies a human readable label for display purposes. The optional type attribute describes the segment type, for future compatibility with Sequence Ontology-based feature typing.

: Each segment has zero or more <TYPE> tags, which summarize the types of annotation available. The attributes are '''id''' (required), which is a unique id for the annotation type and can be used to retrieve further information from the annotation server (see [#feature_linking Linking to a Feature]), '''method''' (optional), which indicates the method (subtype) for the feature type and the '''category''' (optional) attribute, which provides functional grouping to related types. The tag contents (optional) is a count of the number of features of this type across the segment.

===Retrieve the Annotations Across a Segment (features cmd)===

  ''PREFIX''/das/''DSN''/features?segment=''REF:start,stop''[;segment=''REF:start,stop''...]

                                       [;categorize=''yes|no'']

: If specified, the segment argument restricts the list of annotations to those that overlap the indicated range. Each segment argument uses the format format ''reference:start,stop'', where ''reference'' is the ID of the reference sequence used to establish the coordinate system, and ''start'' and ''stop'' are the endpoints of the region to query, inclusive. Multiple segments may be specified.

: Zero or more type IDs to be used for filtering annotations on the type field. If multiple type names are provided, the resulting list of features will be the logical OR of the list. For compatibility with versions 0.997 and earlier of this protocol, servers are allowed to treat the type ID as a regular expression, but this feature is '''deprecated''' and should not be relied on.

: Zero or more category IDs to be used for filtering annotations by category. If multiple categories are provided, they are treated as the logical OR. For compatibility with versions 0.997 and earlier of this protocol, servers are allowed to treat the type ID as a regular expression, but this feature is '''deprecated''' and should not be relied on.

: Either "yes" or "no" (default). If "yes", then each annotation must include its functional category.

: Instead of, or in addition to, '''segment''' arguments, you may provide one or more '''feature_id''' arguments, whose values are the identifiers of particular features. If the server supports this operation, it will translate the feature ID into the segment(s) that strictly enclose them and return the result in the ''features'' response. It is possible for the server to return multiple segments if the requested feature is present in multiple locations.

Annotation servers are only required to return annotations which are completely contained within the indicated segment. Servers '''may''' also return annotations which overlap the segment, but are not completely contained within them. Annotations '''must''' be returned using the coordinate system in which they were requested. For example, if a contig ID was used to specify the segment, then the annotation endpoints must use contig coordinates.

        <TARGET id="id" start="x" stop="y">''target name''</TARGET>

: There is a single <GFF> tag. Its '''version''' (required) attribute indicates the current version of the XML form of the General Feature Format. The current version is (arbitrarily) 1.0 The '''href''' (required) attribute echoes the URL query that was used to fetch the current document.

: The <SEGMENT> tag provides information on the reference segment coordinate system. The '''id''', '''start''' and '''stop''' attributes indicate the position of the segment. The '''version''' attribute indicates the current version of the sequence map. The id, start, stop, and version attributes are required. The optional '''label''' attribute provides a human readable label for display purposes. The optional type attribute describes the segment type, for future compatibility with Sequence Ontology-based feature typing.

: Each feature has just one <TYPE> field, which indicates the type of the annotation. The attributes are '''id''' (required), which is a unique id for the annotation type and can be used to retrieve further information from the annotation server (see [#feature_linking Linking to a Feature]), and the '''category''' (optional, recommended) attribute, which provides functional grouping to related types. The reference server's annotations can consist of additional overlapping landmarks (parents, children, and neighbors), which should be marked "yes" in the third attribute '''reference''' (optional, defaults to "no") to indicate that the feature is a structural landmark within the map (this feature can be annotated). The tag contents (optional) is a human readable label for display purposes.If a '''reference''' annotation has either or both of the optional attributes, '''subparts="yes"''' and '''superparts="yes"''', then in addition to being useable as a reference sequence, the feature contains subparts and/or superparts that themselves can act as reference features. This can be used to reconstruct reference server's assembly. See also [#fetching_assembly Fetching Assembly Information].

: Each feature has one <METHOD> field, which identifies the method used to identify the feature. The '''id''' (optional) tag can be used to retrieve further information from the annotation server. The tag contents (optional) is a human readable label.

; <START>, <END> (required; one apiece per FEATURE)

: These tags indicate the start and end of the feature in the coordinate system of the reference sequence given in the <SEGMENT> tag. The relationship between the feature start and stop positions and the segment start and stop is that the two spans are guaranteed to overlap.

; <SCORE> (required; one per FEATURE)

: This is a floating point number indicating the "score" of the method used to find the current feature. The number can only be understood in the context of information retrieved from the server by linking to the method. If this field is inapplicable, the contents of the tag can be replaced with a '''-''' symbol.

; <ORIENTATION> (required; one per FEATURE)

: This tag indicates the orientation of the feature relative to the direction of transcription. It may be for features that are unrelated to transcription, '''+''', for features that are on the sense strand, and '''-''', for features on the antisense strand.

; <PHASE> (required; one per FEATURE)

: This tag indicates the position of the feature relative to open reading frame, if any. It may be one of the integers , '''1''' or '''2''', corresponding to each of the three reading frames, or '''-''' if the feature is unrelated to a reading frame.

: The target sequence in a sequence similarity match. The '''id''' attribute provides the reference ID for the target sequence, and the '''start''' and '''stop''' attributes indicate the segment that matched across the target sequence. All three attributes are required. More information on the target can be retrieved by linking back to the annotation server. See [#feature_linking Linking to a Feature].

: The <GROUP> section is slightly odd, as it is derived from an overloaded field in the GFF flat file format. It provides a unique "group" ID that indicates when certain features are related to each other. The canonical example is the CDS, exons and introns of a transcribed gene, which logically belong together. The group '''id''' attribute (required) provides an identifier that should be used by the client to group features together visually. Unlike other IDs in this protocol, the group ID cannot be used as a database handle to retrieve further information about the group. Such information can, however, be provided within <GROUP> section, which may contain up to three optional tags.The '''label''' attribute (optional) provides a human-readable string that can be used in graphical representations to label the glyph.The '''type''' attribute (optional) provides a type ID for the group as a whole, for example "transcript". This ID can be used as a key into the [#stylesheet stylesheet] to select the glyph and graphical characteristics for the group as a whole.

:: A human-readable note in plain text format.

:: The target sequence in a sequence similarity match. The '''id''' attribute provides the reference ID for the target sequence, and the '''start''' and '''stop''' attributes indicate the segment that matched across the target sequence. All three attributes are required. NOTE: although this tag is present in the GROUP section, it applies to the FEATURE, and it is preferred to place it directly in the <FEATURE> section. Earlier versions of this specification placed the TARGET tag in the GROUP section, and clients must recognize and accomodate this.

The request for a named segment may fail because: (1) the reference sequence is not known to the server or (2) the requested region is outside the bounds of the segment. In both cases, an exception is indicated. In the case of a reference server, which is expected to be authoritative for the map, the <GFF> section will flag the problem by issuing an <ERRORSEGMENT> tag instead of the usual <SEGMENT> tag. This tag has the following format:<code> <ERRORSEGMENT id="id" start="start" stop="stop"></code>The '''id''' attribute (required) corresponds to the ID of the requested segment, and '''start''' and '''stop''' (optional) correspond to the requested bounds of the segment if this was specified in the request.Unlike a reference server, an annotation server is not required to know the identities of all the segments. Therefore when presented with a segment ID that it doesn't recognize, it can't know whether this is a true client error or merely an unannotated segment. In this case, an annotation server will issue an <UNKNOWNSEGMENT> tag. This tag has the same syntax as <ERRORSEGMENT> but doesn't necessarily imply an error.If an annotation server detects a request for a region outside the bounds of a segment that it has annotated, it will issue an <ERRORSEGMENT> exception.In the case of a request for multiple segments, the server will return a mixture of <SEGMENT> sections for valid segments, and <ERRORSEGMENT> or <UNKNOWNSEGMENT> sections for invalid ones.

<DASGFF><GFF version="1.01" href="http://das.sanger.ac.uk:80/das/pfam/features"><SEGMENT id="P08487" version="d1dfb367c112d4820eeffe4eab1d6487" start="1" stop="1291"><FEATURE id="C2" label="C2:1090-1177"><TYPE id="SO:0000417" category="inferred from reviewed computational analysis (ECO:0000053)">polypeptide_domain</TYPE><START>1090</START><END>1177</END><METHOD id="Pfam">Pfam-A</METHOD><SCORE>1.7e-18</SCORE><NOTE>HMMER Version: 2.3.2</NOTE><LINK href="http://pfam.sanger.ac.uk/family?entry=PF00168">C2</LINK></FEATURE></SEGMENT></GFF></DASGFF>

===<font color="red">We are proposing to remove Linking to a Feature from the spec===

===Retrieving the Stylesheet===

This document is intended to provide hints to the annotation display client. It maps feature categories and individual types to a series of glyphs known to the display client.

       <CATEGORY id="Similarity">

  <TYPE id="NN">

  <TYPE id="NP">

  <TYPE id="PN">

  <TYPE id="PP">

Groups can also have stylesheet entries. If present, they are located in the category named "group". Typically a group will be associated with the "line" glyph, which as described below, draws connections between the members of a group.

===Glyphs and Groups===

Glyphs and their attributes are typically applied to individual features. However, they can be applied to entire groups as well (via the <GROUP> '''type''' attribute). In this case, the glyph will apply to the connecting regions '''between''' the components of the group.

<CATEGORY id="Transcription">

    <TYPE id="exon">

     <TYPE id="utr">

       </LINE>

    </GLYPH>

</TYPE>

  ...

==<font color="red">Fetching Sequence Assemblies We are proposing to deprecate this as not many people use it????==

===<em>component</em>===

===<em>supercomponent</em>===

===<em>translation</em>===

===<em>transcription</em>===

===<em>variation</em>===

===<em>structural</em>===

===<em>similarity</em>===

===<em>repeat</em>===

===<em>experimental</em>===

 

 

 

 

 

 

 

 

 

 

===<em>ARROW</em>===