Stop Designing Fragile Web APIs

My customers will be angry if I break my API.

When you release your Web API, it’s carved into stone. It’s a scary commitment to never make an incompatible change. If you fail, you’ll have irate customers yelling in your inbox, followed by your boss, and then your boss’s boss. You have to support this API. Forever. Unless you version it, right?

After publishing The Web API Checklist, I received comments (#1, #2) regarding API versioning. Before you struggle with how to version your API, I want you to know how to design your API to avoid future incompatibilities.

If my API is well designed, it won’t be fragile.

It is possible to design your API in a manner that reduces its fragility and increases its resilience to change. The key is to design your API around its intent. In the SOA world, this is also referred to as business-orientation. It’s a difficult design concept to understand, best explained with a fictitious example:

What’s the difference between these URLs?

http://api.fbi.gov/wanted?
  order_by=notoriety,desc&
  limit=10&
  page=1&
  fields=name,aka,known_associates,
  reward,description,last_seen

This is a URL for the FBI most wanted list. The API contains many features: you can order by arbitrary fields, ascending or descending; you can specify the result count; you can query page-by-page; and you can specify details to retrieve.

Versus…

http://api.fbi.gov/wanted/most

This is a URL for the FBI most wanted list.

These URLs have the same goal, but they vary in how they accomplish it. The first is a programmer’s design. Programmers know what they can provide, and they give you every feature they can. The design does not describe the intent of the user, but instead defines details about the request. The second URL is very specific about its intent to deliver “The FBI Most Wanted List”, and vague about the details; this is an intent-driven design.

Designing your API with “intent” in mind will reduce fragility.

The intent-driven design has advantages over the programmer’s design:

#1. Easier to Use — No knobs to turn, no details to learn.

#2. Flexible — If the list ordering changes from by notoriety to by public_outcry, the intent-driven API can change server-side.

#3. Consistent — A 24-hour news network can’t sort the list by race then notoriety.

#4. Loosely Coupled — After complaints that “notoriety” is a made-up number, the FBI can hide the field; the intent-driven design is unchanged. (Edit: Whoops, this is a backwards incompatible change; there’s a blunder! If it was important to remove “notoriety” immediately due to a PR kerfuffle, you’re not going to want to provide it in a versioned API either. You could just start returning a fixed value all the time though, which would be backwards-compatible and work well in the intent-driven API.)

#5. Optimizable — Can be calculated when the database is updated, instead of built on-demand. Optimizing the programmer’s design for every combination is much more difficult.

#6. Cacheable — Easily made cacheable. The complexity of the programmer’s design makes caching difficult, (eg. normalizing the query parameters), and ineffective (eg. &limit=5 / 10 / 15 are cache misses).

#7. Easier to Develop — Higher complexity makes it difficult and time-consuming to develop and test the programmer’s API design.

#8. Efficient Validation Model Caching — Quickly check If-None-Match/If-Modified-Since HTTP headers to provide “304 Not Modified” responses.

But, I need a more generic API design…

I hope you’re thinking of a specific reason why you need that generic design. That reason is the intent that allows you to design a better API. For example, the flexible API is great for developing a user interface: it allows sorting by any field, customizable pagination size, and filtering or searching. With that UI intent in mind, you can make design decisions that will provide an easy-to-use, easy-to-develop, flexible, consistent, loosely coupled, optimizable, cacheable, and efficient API. For example:

  • Asked to sort by a field that’s been removed? Meh, ignore it. Interactive users will compensate by selecting a new ordering.
  • API request has a ridiculous pagination size, and a request is like a DoS attack? Max out at a reasonable page size, like 100.
  • Ordering records by an expensive aggregate calculation? Denormalize that field and have it calculated in an overnight job, and add visibility into the staleness for the UI.

Create multiple, specific API endpoints for specific intents, and use that intent to influence the design.

What about DRY (don’t repeat yourself)?…

Don’t repeat yourself in your implementation, but don’t worry about repeating yourself in your API design. If you provide multiple API endpoints to retrieve similar objects with different intent, start with common code paths. Specialize when needed.

More specific services are easier for clients to use, and for you to maintain. Those are the same advantages that DRY gives you.

Does this really work in a real-world API?

Yes, this applies to the real world. An excellent example of this is GitHub’s commit status API, which is used by continuous integration services to mark repository versions as good to merge.

  • Defines specific functionality: assign a state to a revision of a repository.
  • GitHub automatically associates it with pull-requests, and displays the status prettily.
  • No HTML, or customizable states; the API has the minimal amount of data required.
  • Future-proof: it satisfies a defined problem in a minimalist manner.
  • Statuses are associated with revisions; adding more commits to a pull request works like magic.
  • GitHub has flexibility to make changes without breaking compatibility:
    • GitHub could rewrite the entire pull request feature without touching this API.
    • Statuses could be applied to other UIs, like the repository history.
    • The same API could be applied if GitHub became MercurialHub or SubversionHub or PerforceHub or CvsHub or RcsHub.
  • GitHub has flexibility in displaying the commit status, because of the simple information collected:

Another real-world example is Twilio’s AvailablePhoneNumbers API. The intent is to search for a phone number to assign to your account. It looks like a typical collection API, but details that aren’t relevant to the intent are missing, like specifying how many numbers to return or ordering them.

Summary

  • Design your API with a specific intent in mind.
  • Be vague in the details.
  • Provide multiple APIs differentiated by their intent.
  • Reduce code duplication by sharing common implementations, not by providing a generic service.

Please join in the discussion on this article over on Hacker News and reddit.


Did you find this article interesting? I plan to write more on APIs in the future, if you’d like to catch whatever I have to say next, I’m starting a mailing list where I’ll be distributing more API-related content.



Are you missing out on the latest API news, advice, design ideas, and case studies? You might be, unless you’re subscribed to the Web API Design Tips and Tricks newsletter! Subscribers will get the earliest news, access, and discounts for the ApiSlice, the intelligent Web API documentation system!




* = required field



Be Sociable, Share!

6 Comments on Stop Designing Fragile Web APIs

  1. ed
    2014/08/03 at 1:13 pm (1 year ago)

    Hey bro… couldn’t sign up to your newsletter… wanted to be kept in the loop. Check if maybe something is not working?

    ed

    Reply

4Pingbacks & Trackbacks on Stop Designing Fragile Web APIs

  1. The case for keeping dead code | Ryan Kohn
    2013/04/22 at 9:09 am (2 years ago)

    […] When you maintain a completely internal codebase, there is no reason to keep code that’s no longer used. However, if you maintain a library that has an external API, removing code that you believe is no longer used may have a breaking effect on anyone who uses your API. Once you make a contract to have your API work a specific way, it becomes costly to change that. You will need to communicate the change to everyone who uses the API. Even if the method really is no longer used, you don’t want to prevent someone from using it in a perfectly valid way in the future. This is why you must always design your API with its usage in mind. […]

    Reply

Leave a Reply