The art of happiness is to serve all — Yogi Bhajan
Once we get beyond the question of the basic HTTP URI-ness of the digital object identifier (DOI) — since for each DOI there exists DOI-based URIs due to the dx.doi.org and hdl.handle.net proxies, this issue is moot — and old-skool questions of “coolness” based on the relative brittleness over time of creative URI encoding [1], we are then left with the more substantial question of whether DOI-based HTTP URIs really “behave” themselves within the “Web-of-Objects” universe. The purpose of this post is to identify the problem and propose a potential solution, implementation of which will require certain changes to the current Handle System platform. I believe that if the proposed changes are made, lingering questions concerning the “URI-ness” of DOIs (and Handles) will disappear, once and for all.
Note: It is beyond the scope of this post to present all of the gory background details regarding the Handle System, the DOI, and the 1998 and 2008 versions of “Cool URIs.” If there is enough interest in a stand-alone article, I will happily consider writing a longer version in the future, perhaps as piece for D-Lib Magazine.
With the increasing influence of semantic web technologies there has been strong interest in assigning actionable HTTP URIs to non-document things, ranging from abstract ideas to real world objects. In the case of URI-named, Web-accessible physical items — sensors, routers and toasters — this is sometimes referred to as The Web of Things. Until 2005 the community disagreed as to what an HTTP URI could be assumed to represent, but a June 2005 decision by the W3C TAG settled the issue: If a server responds with an HTTP response code of 200 (aka a successful retrieval), the URI indeed is for an information resource; with no such response, or with a different code, no such assumption can be made. This “compromise” was said to have resolved the issue, leaving a “consistent architecture.” [3]
The result of this decision was to force consensus on how to apply the long-established principles of HTTP content negotiation in more consistent ways. In particular, “human” and “machine” requests to a given entity URI — a top-level URI representing a “thing” — should be treated differently; for example, there should be different responses to requests with HTTP headers specifying Accept: text/html (for an HTML-encoded page) versus Accept: application/rdf+xml (for RDF-modeled, XML-encoded data). This is most often seen in the semantic web and linked data worlds, where it is now common to have both textual and machine readable manifestations of the same URI-identified thing.
Modern web servers including Apache have been engineered to handle these requests through content negotiation [4]. Through standard configuration procedures, site administrators specify how their servers should respond to text/html and application/rdf+xml requests in the same way they specify what should be returned for alternate language- and encoding- requests; “en,” “fr,” etc. Typically, when media-specific requests are made against entity URIs representing concepts, the accepted practice is to return a 302 Found response code with the URI to a resource containing a representation of the expected type, such as an html-encoded page or an XML document with RDF-encoded data.
Many readers of this post will be familiar with the basic idea of HTTP proxy-based Handle System name resolution: A HTTP resolution request for a DOI-based URI is made to a proxy — a registration-agency run proxy such as dx.doi.org or the “native” Handle System proxy hdl.handle.net — the appropriate local handle server is located, the handle record for the DOI is resolved, and the default record (e.g. a document information page) is returned to the client as the payload in a 302 Found response. In a Web of Documents this might make sense, but in a universe of URI-named real-world objects and ideas, not so much.
The 2008 document provides two requirements for dealing with URIs that identify real world objects:
- Be on the Web: Given only a URI, machines and people should be able to retrieve a description about the resource identified by the URI from the Web. Such a look-up mechanism is important to establish shared understanding of what a URI identifies. Machines should get RDF data and humans should get a readable representation, such as HTML. The standard Web transfer protocol, HTTP, should be used.
- Be unambiguous: There should be no confusion between identifiers for Web documents and identifiers for other resources. URIs are meant to identify only one of them, so one URI can’t stand for both a Web document and a real-world object.
In the post-2005 universe of URI usage as summarised above and detailed in [2], if DOI-based URIs are used to represent conceptual objects these rules will be broken! For example, Handle System proxies today cannot distinguish between Accept: codes in the request headers; the only possible resolution is the default (first) element of the Handle record. (For hackers or merely the curious out there, I encourage you to experiment with curl at your command line or Python’s urllib2 library, hitting the DOI proxy with a DOI-based URL like http://dx.doi.org/10.1109/MIC.2009.93.) This problem with how proxies resolve DOIs and Handles is a lingering manifestation of the native Handle System protocol not being HTTP-based and the system of HTTP-based proxies being something of a work-around, but the vast majority of DOI and Handle System resolutions occur through and rely on these proxies.
One possible solution would be to enable authorities — Registration Agencies — who operate within the Handle System to configure how content negotiation within their Handle prefix space is handled at the proxy. For document-based use of the DOI an example of this would be to return the URI in the first element of the Handle record whenever a text/html request is made and (for example) the second element whenever an application/rdf+xml is made. When a request is made to the proxy, request-appropriate representation URIs would be returned to the client along with the 302 Found code. This approach treats the DOI-based URI as a conceptual or entity URI and gives the expected responses as per [2]. pax vobiscum…
Readers familiar with the Handle System will appreciate that there are many potential schemes for relating HTTP content type requests to elements of the Handle record; in the example above I use position (index value), but it is also possible to use special TYPEs.
Handle servers are powerful repositories and can implement potentially many different models other than redirection as described above. Sometimes, for example, the desire is to use a Handle record as the primary metadata store. In that case, the preferred application/rdf+xml might very well be to return an RDF-encoded serialisation of the Handle record. How this is handled should be a feature of the Handle server platform and a decision by registration agencies based on their individual value propositions, and not locked in by the code.
I eagerly look forward to your comments and reactions on these ideas!
Update 1: In a comment to this post, Herbert Van de Sompel argues that the real question is, what should DOIs represent? Herbert asserts that DOI-based URIs should model OAI-ORE resource aggregations and that Handle System HTTP proxies should behave according to OAI-ORE’s HTTP implementation guidelines. Herbert’s suggestion doesn’t conflict with what I’ve written above; this is a more subtle and (arguably) more robust view of how compound objects should be modeled, which I generally agree with.
Here’s how OAI-ORE resolution would work following the Handle proxy solution I’ve described above: Assume some DOI-based HTTP URI doi.A-1 identifies an abstract resource aggregation “A-1” (In OAI-ORE nomenclature doi.A-1 is the Aggregation URI). Following the given HTTP implementation example, let there be two Resource Maps that “describe” this Aggregation, an Atom serialization and an RDF/XML serialization. Each of these Resource Maps is (indeed MUST be) available from different HTTP URI’s, ReM-1 and ReM-2, but the desired behaviour is for either to be accessible through the DOI-based Aggregation URI, doi.A-1. Let these two URIs be persisted in the Handle record, preferably using TYPEs which distinguish how they should be returned to clients based on the naming authority’s configuration of the HTTP proxy. By the approach I describe above, the Handle System proxy would then respond to resolution requests for doi.A-1 with 303 See Other redirects to either ReM-1 or ReM-2 depending upon MIME-type preferences expressed in the Accept: headers of the requests.
Update 2: Complete listing of MIME types for OAI-ORE Resource Map serializations. Follow-up conversations with Herbert Van de Sompel, Carl Lagoze and others have reminded me I neglected to mention how the OAI-ORE model recommends handling “HTML” (application/xhtml+xml and text/html) requests! This is not a minor issue, since the purpose of ORE is to model aggregations of resources and not resources themselves, and so it is not immediately clear what such a page request should return. My solution (for the purposes of this blog post) is for Handle System HTTP proxies to respond to these requests also with 303 See Other redirects, supplying redirect URIs that map to appropriately-coded “splash screens.”
For completeness, the table below (repeated from [5]) lists the standard MIME types for Resource Map serializations. Continuing with the major theme of this post, Handle System HTTP proxies resolving requests for DOI-named ORE Resource Maps should follow these standards so the clients may request appropriate formats using HTTP Accept: headers.
| Resource Map Type | MIME type |
|---|---|
| Atom | application/atom+xml |
| RDF/XML | application/rdf+xml |
| RDFa in XHTML | application/xhtml+xml |
If a client prefers RDF/XML but can also parse Atom then it might use the following HTTP header in requests:
Accept: application/rdf+xml, application/atom+xml;q=0.5
The table below list the two common MIME types for HTML/XHTML Splash Pages following the W3C XHTML Media Types recommendations.
| Resource Map Type | MIME type |
|---|---|
| XHTML | application/xhtml+xml |
| HTML (legacy) | text/html |
Thus, if a client wishes to receive a Splash Page from the Aggregation URI and prefers XHTML to HTML then it might use the following HTTP header in requests:
Accept: application/xhtml+xml, text/html;q=0.5
As noted in [5] there is no way to distinguish a plain XHTML document from an XHTML+RDFa document based on MIME type. It is thus not possible for a client to request an XHTML+RDFa Resource Map in preference to an RDF/XML or Atom Resource Map without running the risk of a server correctly returning a plain XHTML Splash Page (without included RDFa) in response.
The Handle record for a given DOI or Handle identifying an ORE aggregation would therefore contain a set of URIs reflecting the mappings in the tables above. A content-negotiation-savvy Handle System HTTP proxy would then return the appropriate URI in the 303 Found response, based on its configuration and policies.
Update 3 (28 Apr 2011): Great news! This week CrossRef, the IDF and CNRI announced the completion of their implementation of Content Negotiation for CrossRef DOIs. As that post describes, it is an implementaton of “Option D” in last year’s CrossTech post, DOIs and Linked Data: Some Concrete Proposals. Ed Summers provides a great explanation of the significance of this in his recent INKDROID post, DOIs as Linked Date.
Update 4 (18 Feb 2012): It occurs to me that it might be instructive to readers to see Content Negotiation for CrossRef DOIs in operation, using curl at the command line. Here are the header responses for the DOI-based URI given earlier:
$ curl -H "accept: application/rdf+xml" http://dx.doi.org/10.1109/MIC.2009.93 -L -I HTTP/1.1 303 See Other Server: Apache-Coyote/1.1 Location: http://data.crossref.org/10.1109%2FMIC.2009.93 Expires: Sun, 19 Feb 2012 13:13:00 GMT Content-Type: text/html;charset=utf-8 Content-Length: 178 Date: Sat, 18 Feb 2012 20:55:42 GMT HTTP/1.1 200 OK Date: Sat, 18 Feb 2012 20:55:44 GMT Server: Apache/2.2.3 (CentOS) X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.7 Vary: Accept Content-Length: 11876 Status: 200 Connection: close Content-Type: application/rdf+xml
You can see that the initial request to the generic DOI HDL proxy is referred over to CrossRef, which acknowledges that can respond with the RDF content type requested in the Accept: header. An identical request without the trailing -I would get you the RDF serialized in XML.
References:
- [1] Tim Berners-Lee, Cool URIs don’t change (1998)
- [2] Leo Sauermann, et.al., Cool URIs for the Semantic Web (2008)
- [3] Tim Berners-Lee, What HTTP URIs Identify (2005)
- [4] Apache Foundation, Content Negotiation (Apache 2.2)
- [5] Carl Lagoze and Herbert Van de Sompel, et.al., Common MIME Types for Resource Maps and Splash Pages, ORE User Guide: HTTP Implementation (17 Oct 2008).
Bitwacker Associates 
John – I already mentioned on Twitter. The answer is at http://www.openarchives.org/ore/1.0/http.html#RedirectCN. Apart from that I think your question is too centered around handle system technicalities. I am not sure too many people care about that one. The core question is how are you modeling what DOIs represent on the web. My answer is http doi identify ORE Aggregations. I think that is the DOIs entry point into the linked data and sem web world.
By: Herbert on February 4, 2010
at 11:09 pm
Hi John:
This is a real nice post on a very timely issue. And thanks for both the background and the solution you propose.
I am a little confused in one respect though. It seems to me there are two separate (but related) points at hand: 1) a mechanism (e.g. status 303/hash URI) to relate URI for RWO and URI for associated description, and 2) a mechanism (e.g. HTTP conneg) to provide alternate machine/user descriptions. I get the feeling that these two are being conflated somehow and that more prominence is being given #2. In fact, I’m not strictly sure that a machine readable description is actually *required*, although is very highly desirable.
So, if one just falls back on the default (“human”) description provided by the regular HTTP dereference of an Information Resource, and focuses on the first point.
The TAG resolution on httpRange-14 spells out very clearly what a 303 resolution is for (“any resource”), but remains quiet on other 3** codes. Interesting to note that RFC 2616 has this to say on 302:
“However, most existing user agent implementations treat 302 as if it were a 303 response, performing a GET on the Location field-value regardless of the original request method.”
This is obviously the sense in which current DOI HTTP URI is being used. But still it returns a 302 and not a 303 (albeit intending 303). But httpRange-14 says nothing about 302 per se.
Tony
By: Tony Hammond on February 8, 2010
at 10:41 am
Hi John,
Thanks for this article. The clear and thorough explanation is appreciated. However the big question in my mind is: why is the handle proxy a better place for the negotiation than the endpoint URL?
It seems like the owner of the resource would be a better negotiator.
Thanks,
Sean
By: Sean Reilly on February 8, 2010
at 4:32 pm
Thanks for the kind worlds!
RE your question, I believe having the “owner” of the “resource” do the content negotiator misses the point of what the DOI (and the DOI-based HTTP URI) represents, and more importantly what clients and services THINK HTTP URIs “mean.”
You don’t want clients to require special knowledge about the HTTP URI! If the universe is forced to treat DOI-based HTTP URIs that name data, things or documents differently than other HTTP URIs representing the same things, then there is a problem. This is the situation today (I believe) with how the HS HTTP proxy handles content negotiation. If however the RA/administrator has the option to return different types for different Accept header field values, then the DOI-based URIs become indistinguishable from “normal” HTTP URIs.
By: John Erickson on February 8, 2010
at 4:59 pm
I might be missing something, but I don’t see how Sean’s proposal requires any special knowledge about the HTTP URI on the part of the client.
If a DOI 10.example/foo represents a resource which can be accessed as either HTML or RDF, then one could have http://dx.doi.org/10.example/foo redirect to http://example.com/foo which *itself* returns the appropriate type (or even another redirection) based on the Accept: header. From the client’s point of view, if you ask for HTML you get HTML, if you ask for RDF you get RDF. That’s no different, from the point of view of the client, from having http://dx.doi.org/10.example/foo redirect to http://example.com/foo.html in one case and http://example.com/foo.rdf in the other.
Thanks,
Robert
By: Robert Tupelo-Schneck on February 8, 2010
at 5:31 pm
(This is a response to Robert Tupelo-Schneck’s comment on February 8, 2010 at 5:31 pm)
I argue that this *does* miss the point. Why should the client double-pump (to use a football analogy)? If the client “wants” the server — it doesn’t know for DOI — to provide RDF+XML (for example), why shouldn’t the HTTP proxy turn around and give it the URI for the appropriate representation? You’re suggesting that the proxy return first something it didn’t ask for, then re-negotiate, and then get what it wants.
My larger point is, I believe there is an opportunity here to employ the HS (and the DOI) in a way that we’ve been talking about for (in my case) about 14 years — really being entity identifiers for multi-faceted objects and not expensive URL proxies…If all the proxy does is pass off content negotiation (when in fact it could be much smarter), in the coming Web of Objects the HS risks looking less attractive…
By: John Erickson on February 8, 2010
at 5:51 pm
Hi John,
I’m not (yet) arguing against the proxy being smarter about content negotiation, however I’m still not sure what makes it better than the endpoint URL doing the negotiation.
I can see a few different ways for the Handle/DOI proxy to be in the loop when doing content negotiation:
1) The proxy is a simple handle-to-URL resolver and the resulting URL (example.com in Robert’s example) takes care of content negotiation. This seemed appropriate to me because the URL’s owner could return either XML+RDF directly or a redirect to a separate RDF URL.
2) The proxy looks for a requested content-type in the request and uses it to determine if the request should be redirected to a different URL in the resolved handle. This has either the same or one fewer “bounce” than method #1 above but also requires that the content owner register different URLs in their handles, as opposed to managing the negotiation locally in their own HTTP server.
3) The proxy looks for a requested content-type in the request, extracts the RDF data directly from the handle values and returns it to the user. This doesn’t require any HTTP communication apart from the initial proxy request. But it does require pushing potentially lots of data into the handle system where it is likely harder to manage than the if it were within the content owner’s HTTP server.
I assume that nobody is interested in option #3 and that you are suggesting option #2 because option #1 is already there. While on one hand I prefer to keep the proxy as a simple and fast tool, it is probably pretty easy to enable option #2 using the relatively new multiple-resolution handle value which already does a sort of (location, not content) negotiation.
Thanks,
Sean
By: Sean Reilly on February 8, 2010
at 10:11 pm
Made a general response but it got published as a reply further up in the thread (before last two messages). – Tony
By: Tony Hammond on February 9, 2010
at 6:13 am
(This is a response to Sean’s post of February 8, 2010 at 10:11 pm…)
I propose that we think about it in the following way (terms are from RFC 2295):
1. The digital object named by the HDL (and thus the HDL-based HTTP URI) is a “transparently negotiable resource” — a resource which has multiple representations (variants) associated with it
2. The HDL record contains as individual elements, but is not limited to, a “variant list” of the URIs of representations bound to that object
3. The variants are distinguished in the HDL record by different TYPEs; for the purposes of this discussion, these TYPEs indicate content-type (MIME type) variants, but could also indicate snapshots in time or other bases for user agent selection. See esp.
4. To support “conneg” the administrator (e.g. the registration agency, or RA) creates and maintains on the proxy a mapping of TYPEs to conneg criteria, especially content-types.
5. These mappings may be highly customized; for example, some administrators may wish to support fine-grained differentiation of MIME types; some might want to support language types; some might specialize in supporting well-behaved responses to HDLs/DOIs naming OAI-ORE-style aggregations.
6. This approach is not obviously compatible with using a HDL server as an immediate repository. A different configuration scheme would be required, say, to return (selected) contents of the HDL record serialized as RDF/XML or RDF/N3, say.
John
By: John Erickson on February 9, 2010
at 8:35 am
Hi:
Not a direct response to John’s post but just to announce a related post of mine on CrossTech: ‘DOI: What Do We Got’ http://bit.ly/bnCkrn (Aiming with this post just to put something together visually as an aid to understanding.)
Cheers,
Tony
By: Tony Hammond on February 9, 2010
at 10:03 am
Just to be clear, there’s no technical obstacle to implementing this. I suspect that, rather than using multiple type-value pairs as John suggests, we would use a single type-value pair where the value is structured data (a bit of XML, say) which indicates how to redirect depending on the Accept: header or other factors. In fact as Sean mentioned we’d probably want to use a type we’ve already been working on which indicates how to redirect based on client geolocation, query parameters, or other factors. All that’s just implementation detail however.
I would like some further explanation of Tony’s complaint against Sean’s #1, which, after all, works today. It’s true that the existing setup presumes one default URL for redirection, but it doesn’t have to be HTML; it can be any URL returning an entity of any content type, or even a single URL returning one of multiple content types based on content negotiation.
I understand that there is one more redirect involved, and that from the standpoint of having the handle be the primary identifier for an object it is appealing to have the handle itself understand the various representations rather than push that off another level. For completeness, anything else?
By: Robert Tupelo-Schneck on February 9, 2010
at 11:25 am
Sorry, Robert, my reply got shuttled up the thread again. See above at
By: Tony Hammond on February 9, 2010 at 11:34 am
Sorry again, about that.
By: Tony Hammond on February 9, 2010
at 11:36 am
I don’t see how #1 is a tenable position. An RA member should not be able to usurp the baseline semantics set by the RA community. CrossRef, for example, has always said that DOIs for journal articles are for abstract works. And that from a Linked Data perspective (and following TAG ruling on httpRange-14) would make the correct response be a 303, regardless of whether there is HTTP conneg to support parallel HTML, RDF/XML representations.
Seems that if there were an RDF/XML pointer in the handle record that could be used to honour an Accept header request as per option #2.
But with the existing setup there is one default URL which is presumed (required, I think) to be an HTML version and which a 303 could redirect to in absence of conneg machinery or alternate URL records. That would have the merit of clearly signaling to the HTTP community the semantic intent – that there is a complex object (or RWO) at the end. Now that might be something that an RA proxy server should provide rather than a dumb HS. But it should not be something that the content publisher provides – would mean one content provider is dealing with RWO’s or Non-Information Resources, and another with bog-standard Information Resources.
Would really like to see handle keep better aligned with HTTP. We need a more “graceful” interface between the two. And that might well mean support for HTTP conneg.
Cheers,
Tony
By: Tony Hammond on February 9, 2010
at 6:09 am
Readers: Tony’s comment is a response to Sean’s “February 8, 2010 at 10:11 pm” post…Wordpress’ comment nesting is a bit confusing (“too clever by half…”)
By: John Erickson on February 9, 2010
at 7:49 am
Hi Robert:
“I would like some further explanation of Tony’s complaint against Sean’s #1, which, after all, works today.”
Would question whether it does work. Yes, in terms of getting some bits from here to there – i.e. as a blind redirect. But as a semantic operation which aligns with current web architecture – maybe not.
From my point of view the alternate (i.e. machine readable) descriptions (accessed via HTTP conneg) are secondary to the establishing of a correct naming architecture (URI) and service response (HTTP status codes). What is
http://dx.doi.org/10.1234/567
Is it an information resource – per AWWW – or not? TAG says httpRange-14 fixes that by allowing 303 to indicate that it is a “See Other”, i.e. this is URI for a RWO (real world object – a nono-information resource).
But what do we return but a 302. Semantically we’re lost.
Don’t see why – even without the RDF/XML (which of course we would like) – we couldn’t make progress by redirecting to HTML on 303 rather than on 302.
Is all. 🙂
Tony
By: Tony Hammond on February 9, 2010
at 11:34 am
Do I understand you Tony as saying that if the proxy returned a 303 instead of a 302, it wouldn’t matter to you whether any conneg was done at the proxy or one step removed at an URI returned (via 303) by the proxy?
By: Robert Tupelo-Schneck on February 9, 2010
at 12:00 pm
I think that what Tony is saying that the current
302is simply wrong by “modern” standards, and a first, minimal step should be to respond instead with a303with the default response URI.I’m not sure what the point is of the HS having multiple resolution if it can’t be leveraged esp. via conneg at the HTTP proxy. Given a DO that aggregates multiple representations (URIs persisted in the HDL record), what is the conceptual problem with enabling the agent and the server to negotiate over the available variants using the established and generally-accepted methods?
Doing this ups the value proposition of the Handle System model significantly. If all you expect the proxy to do is hand over a URI for which the agent must further negotiate, what’s the point?
By: John Erickson on February 9, 2010
at 12:21 pm
Indeed, I completely agree; since we are already interested in multiple resolution at the handle level it is the right thing to do to support content negotiation at that level. I apologize if I was not clear about that. I’m just interested in understanding all the reasons why the handle level is the right level, instead of at a content owner’s HTTP server.
By: Robert Tupelo-Schneck on February 9, 2010
at 12:33 pm
I guess we agree to agree then.Probably a confusing fatcor here is that in the #-free-HTTP-URIs’-only-name-documents school of HTTP thought, what you get back from a GET is not just the URI owners view of the world, but an authoritative serialization of named the thing itself. And that stronger notion of authority has leaked out a bit into dialog with other perspectives on HTTP, and is being applied to situations where we’re dealing with annotational descriptions rather than authoritative representations-as-in-serializations.
By: Deon on May 24, 2012
at 7:03 pm
[…] Update: if this topic interests you, and you want to read more about it, definitely check out John Erickson‘s blog post DOIs, URIs and Cool Resolution. […]
By: DOIs as Linked Data | inkdroid on October 28, 2013
at 10:39 am