The Emperor’s New API

July 9th, 2009

Note: this is being cross-posted, with additional content, from the Adobe Experience Design site, Inspire.

Strategists everywhere are pushing companies to add APIs to their online properties. Job postings for web developers and architects invariably require experience working with and developing APIs. It comes up in sales meetings, pitches, and casual conversations with industry folks all the time. I’m often left wondering if the term “API” has lost its real meaning and become sort of a stand-in for a handful of fuzzy concepts.

REST APIs are Buzzworthy

In the API chatter, there’s a huge focus on RESTful APIs. Everyone wants a RESTful API. Web developers love REST architectures because the fixed semantics and stateless mapping of resources to URIs lowers the barrier to entry for integration tasks. Devs no longer have to struggle to understand proprietary architectural touches. No WSDL files need to be parsed. No expensive, scale-resistant state-managing caches need to be added. The URI we use in the browser is the same URI our apps use to read JSON or XML, etc. It’s a glorious thing, REST. I absolutely love the style, and preach the benefits to clients whenever possible.

Another thing I peach, when I can, sounds scandalous at first. That message: If you’re adding a REST API to your site, you’re probably doing something wrong. Maybe.

REST APIs are Everywhere. Right?

All the big sites are exposing REST APIs. Twitter has on. Flickr has one. Netflix. Microsoft SharePoint. Everyone, man. Everyone.

But how many are as RESTful as they could be?

If you see the value in a REST API, the last thing you want to do is create something separate from your “main” site or service. Your site is your “API” if you’re being RESTful. If you force a separation or bolt on a separate chunk of URIs meant for API clients you’re probably wasting resources and muddying your domain.

With REST, you should have a distinct, stateless URI for each conceptual resource you provide and should respond to the appropriate HTTP verbs (most commonly, GET, PUT, POST, DELETE, HEAD, TRACE, and OPTIONS).

For example, if you have the concept of a user profile for someone named “TobyJoe” you should have a single URI that represents that profile and that URI should serve any supported format a client application requests. It doesn’t matter what the URI looks like, but there should only be one way of accessing that conceptual resource. You can serve up XML, JSON, HTML, plain text, SVG, or any combination of formats you want. HTTP and the REST style already accommodate for this using certain HTTP request headers. This problem has been solved since the Web existed (basically). And yet, we see a lot of big sites offering two distinct sets of URIs: one for their “site” traffic and one for their “API” traffic.

Really, it shouldn’t matter if someone is requesting the resource at a URI from Safari, Tweetie, cURL, a Flash RIA, or any other HTTP client. The REST style can cover all the content switching and access management you need and you can have a single set of URIs, a single set of semantics, a single engineering team, and a single set of resources to support. The user experience for your third-party development partners will be improved as well.

Twitter: An Easy Example

So why are there two URIs for my Twitter status feed? One for the Twitter-branded HTML (with great microformat support!) at http://twitter.com/tobyjoe and one for XML at http://twitter.com/statuses/user_timeline/tobyjoe.xml?

It’s the same data, in different formats. It’s the same concept: status messages for tobyjoe. Most of the formats Twitter supports are available at the http://twitter.com/statuses/user_timeline/tobyjoe.{EXTENSION} address. Why not the HTML version with the sweet microformats?

Note: I’m sure Twitter has many reasons for this separation. Legacy clients, their ever-changing architecture and unprecedented growth, organizational investment in this style. They have an amazing team over there. I’m only using Twitter as an example because so many developers are familiar with their API.

Still, as both a user and client developer, I’d prefer to see a single URI (or at least one URI template) for each conceptual resource rather than one URI for “people” and one URI template for “machines”.

It’s Not Always That Easy

I know that there are lots of hurdles with development around live products. I don’t discount the operational challenges. I don’t necessarily chastise folks who stray from REST. I would love to see the concept of the RESTful API default to a slightly different meaning that what I see on the web today. Instead of saying, “We are going to add a REST API to our site!” I’d love to see companies say, “We are expanding the choice of representation formats and verbs for some of our URLs!” Not as sexy sounding, but way sexier.

  • I came across a great post by the guy working on Recess (PHP framework) that talks about many similar, and some of the same, topics: Unacceptable Browser Accept Headers

    Good stuff!
  • Hey there, Matthew. Thanks for the comment.

    I want to clarify that I, also, don't give a rat's arse what the URL looks like. Rather, I care that a conceptual resource only have one URI for all representations if the resource is described as RESTful. Otherwise it's misleading and makes generic client libraries hard to develop and obfuscates the set of available resources and representations.

    I don't think interlinking is important, necessarily. Rather, the parts of Dr. Fielding's thesis (ch05) I'm referencing herein has no requirement that one representation link to another representation. When content negotiation is used, a link actually can't specify alternate representations, as the URI would be the same and the headers would be used for the negotiation.

    As for "...doing it wrong...maybe." I think the phrase is as helpful as I wanted it to be. That is, it speaks a truth: bolting on new URIs or a new namespace (like api.* subdomains) does stray from one of the major benefits of RESTful design: one URI per concept, with some subset of HTTP 1.1 verbs and some subset of available content types available.

    To reiterate: I don't care what a URI *looks like* and consider URIs opaque. I just want one URI per resource. To use my Twitter example, I want the URI for my most recent 20 statuses (the concept) to be available at the same URI (or URI template) no matter what representation format I choose, if possible.

    As Nick described, there can be strong reasons to stray from the "one URI per resource" rule – usually due to a particular representation (usually the HTML representation) being a clusterfuck of uncacheable, state-based supplemental data. Still, the default, if a system claims to be RESTful, should be to follow to the basic rules.

    Adding URIs for XML and JSON chunks over HTTP is not necessarily RESTful, despite the splintering the term has taken after the Rails bloggers got ahold of it.

  • The Emperor?s New API #programming http://bit.ly/17SMcp


    This comment was originally posted on Twitter

  • Matthew
    Personally I don't think Roy Fielding would give a rat's arse what exactly your feed URL looks like, provided there's a structured LINK from the html page to the feed resource which spiders can understand and follow.

    The linkability part of the restful style seems more important than semantic debates about what constitutes a separate resource and what constitutes a representation of an existing resource. This seems under-specified and not worth worrying overly about. Certainly phrases like "you're doing it wrong" aren't helpful.

  • The Emperor’s New API: http://bit.ly/11lUnz


    This comment was originally posted on Twitter

  • Ha!

    I mostly map it directly to Fielding's thesis and ignore damn near everything after that (sort of). I think a representation can often be pretty different among formats without impacting the style. I mean, an SVG or PNG representation of a product and an XML representation of the product attributes could be the same concept and same URI. Or not. I think it depends. The loophole/power is definitely in the concept you're conveying. It's where the art can come in... or the abuse!

    And I think all views can be "REST views" in that the REST style isn't about the contents of the representations as much as the roles of components/connectors/resources/representations and the transfer mechanisms and rules about the verbs and the mapping of concepts to URIs...

    I am not as dogmatic about the choice, but:

    1. I do love when I have a truly REST-compliant system to work with
    2. If you say it, do it and have it make sense, or I will curse you. And there is nothing worse than a tobyjoe curse, obviously!
  • That's a great point Toby. I think a broader question is if the REST view should return the *exact* same content as the HTML view.

    On an e-commerce site, the "people who bought this also bought" list on a product page makes sense in the HTML view. Does it make sense in a REST view? (maybe it doesn't)

    Do attributes and their relevance differ for different views of the same model? (I mean this is clearly the case for the .html view. For example, most places don't output things like primary keys on their HTML view.)

    I fear that once we codify "REST-compliant" it will morph into what has happened to "agile" where we now have IBM waterfall type-guidelines for "Agile Certified Development."

    Maybe it's like the Supreme Court with porn. I know its REST when I see it.
  • I've struggled with the load question as well. I've always thought that all the URLs should be pure, although I usually try to write code to respect extensions if a client app is really insisting. Nick, you could actually write this in Restlet pretty easily, but then you have to put a java webserver in front of your apps and you lose all the configurability of an nginx or apache. I'm also guessing that no matter how good NIO is that restlet/grizzly can't match nginx. It's easy enough to test if you can figure out the nginx config (I googled and the results made me want more coffee), I'll write up a restlet front.
  • Nick I think it depends on how you cache the (presumably) more dynamic "site" content. If your representations are highly cacheable (which is a big goal with REST), the profiles might only differ in the shape of the throughput. Not that it's unimportant, but it's probably simple enough to farm out.

    There's also a possible loophole for *some* highly dynamic "site" resources: treat them as a different concept. Don't offer alternate representations if those representations are of naturally different concepts.

    If you're showing a chunk of XML for a product on your store at URL_A, but the HTML representation includes lists of recommendations, aggregates from related content objects, etc, you're *probably* modeling different concepts.

    Most of the time, in my experience, a very divergent "site" representation (aka, dot-html) is more a sign that an alternate format isn't appropriate.

    I don't want to make the case that all concepts should support all representation formats through conneg or anything - just that, if you do choose to support multiple representations of the *same* resource, the default should be to use the same URI.

    Or, don't call it REST-compliant?

  • Good article about #RESTful APIs and why some of them are not RESTful ;) http://bit.ly/11lUnz


    This comment was originally posted on Twitter

  • Good article about #REST http://bit.ly/11lUnz


    This comment was originally posted on Twitter

  • Good article on REST http://bit.ly/11lUnz


    This comment was originally posted on Twitter

  • I think the biggest problem with actually having one URL is partitioning the load for REST APIs and your "Real" site.

    Presumably, you don't want your entire front-end site to go down if an errant developer floods the API with lots of calls. Imagine if Facebook did this and their fbconnect API gets overloaded...

    I guess you can partition based off header, but that becomes a performance nightmare (in terms of Apache or something doing the routing to different server farms.)

    Also, I've found that there's a pretty striking difference between the performance profiles of a web site and a REST API. (And, it's inevitably impossible to optimize for both.)

    Like the idea in theory though. And certainly the same app can (and should) power both...

  • Some thoughts on rethinking REST APIs: http://bit.ly/11lUnz


    This comment was originally posted on Twitter

  • RT @tobyjoe: Some thoughts on rethinking REST APIs: http://bit.ly/11lUnz (via @hyperradiant) REST is another much abused buzzword, like AJAX


    This comment was originally posted on Twitter

  • RT @tobyjoe: Some thoughts on rethinking REST APIs: http://bit.ly/11lUnz


    This comment was originally posted on Twitter

  • Some thoughts on rethinking REST APIs: http://bit.ly/11lUnz


    This comment was originally posted on Twitter

blog comments powered by Disqus