Documenting Hypermedia APIs

Editor's Notes

• #2: Hi, I’m Nick, I’m in charge of API documentation and developer experience at Worldpay, I’m going to talk a little about how we’ve documented a new suite of hypermedia driven APIs. I touched on this very briefly at the previous API the docs conference in Paris in a short cameo but I’ll try and explain a little more today.
• #3: I’ll start by giving a brief introduction to hypermedia and hateoas. Just out of interest, can I get a quick show of hands who has worked with hypermedia driven APIs. I’ll try and keep it fairly high-level. Feel free to grab me afterwards if you need a little more detail. I’ll then run through why, at Worldpay we decided to go with hypermedia, before digging in to our documentation and some more visual assets we have built. Finally I’ll touch on where we want to go next and some areas we want to focus on for improvement. I’ll try and avoid starting too much of a debate on API design, we think this works for us but it will not be for everyone.
• #4: Hypermedia Sooo…hypermedia. This is actually a fairly prevalent concept within REST APIs. Hypermedia generally refers to anything that isn’t simply hypertext, this could be images, video, graphics etc. In this context we will talk about in terms of hyperlinks. HATEOAS HATEOAS, or hypermedia as the engine of application state is essentially a system that is driven by hypermedia. This allows the consumer to manage state using hypermedia links returned in responses. I’ll talk a little bit more about how this works for us in my next slide. HAL HAL, or Hypertext application language is a convention that we use to standardize all the responses. This ensures the client will always be able to consume the hypermedia links in the same way.
• #5: So he’s a couple of diagrams designed to give a VERY SIMPLE view of the differences between a more traditional REST API and a hypermedia driven API. With REST you can see that all resources are considered equal. You can make requests to any of the resources at any time because all of the URLs are known to you. This means that it is stateless. With a hypermedia driven API, you can see that the state is managed through the links. The only URL you need to be aware of is the URL for the root resource (the one furthest left). The URLs for the ‘twigs’ on the right of the diagram are not known up front, these are ALL generated dynamically. In order to access those resources, you’ll need to use the hypermedia link returned to you by the API. This means that we could change these URLs at any time if we wanted to. Instead of thinking about these resources as URLs we prefer to think about them as actions, you need only know what your next action is and the API will handle the URLs dynamically for you.
• #6: So why did we decide to go with a hypermedia driven API at Worldpay? Firstly from a payments perspective the lifecycle of a payment is very much suited to this. In the past with a more traditional approach, we could receive lots of “invalid” requests as the state of a payment was not managed by the system. I say invalid as from the clients perspective this is a valid request. For us this could cause issues downstream and also result in unexpected behavior for the client. Let’s take a simple payment as an example. This is very much a distilled down version of what goes by the way. The first step is to authorize the funds, the next step is to settle and, if there’s an issue with the product you might need refund the payment. Obviously, it would be silly to attempt to refund a payment before it’s even been made, so we just don’t give you a URL for that. The only way to do that is with the encoded link returned to you once you’ve settled the funds. Once that payment cycle is complete, you go back round to that start and through the process again, those URLs you had earlier are no longer required, you need to go and generate some new ones by creating a new payment. The other reason for hypermedia is the notion of discoverability, the client can navigate their way through and discover their next actions by taking a look at the links in the responses. This is a little alien for us when it comes to docs, as we’re used to providing everything up front. It helps ease that barrier of entry and hopefully reduces a bit of complexity. Data showed lots of incorrect requests – bad Can’t refund a payment that doesn’t exist. Therefore not provided with the refund URL Discoverability helps to map out flows of requests Better alignment for dev teams
• #7: Right, on to the actual documentation! It’s pretty standard nowadays to use an API specification to support documentation and other assets. Unfortunately for us OpenAPI doesn’t yet support a fully hypermedia driven API. It’s much more suited to the more static API design and doesn’t support URLs that are generated dynamically. This made me sad, I’ve been a big supporter of the swagger/open API and I’m very much bought in to all the value it brings. So what did we to solve this? Well we just decided we would create our own spec… Yes we are a little bit mad but for us there’s so much value in having that API spec, we’ve actually used it to rapidly design future APIs we like to be spec first. We have lots of different development teams working on lots of different APIs so it really helps to keep them all aligned. We make them responsible for keeping them up to date and we hope to start automating the generation of parts of the specifications.
• #8: Okay, what have we actually produced by way of documentation? As mentioned, we’ve created our own specification files which are linked to by the API, as per the hal specification and they can be browsed and downloaded. We’ve used these to generate our API reference docs and we have a more traditional user guide to run through concepts and to help educate our developers around best practices for integration
• #9: The specification for our APIs is made up of a number of json files. One for each resource in fact. The first and arguably the most important is our resource tree. This defines how the resources are linked together. This shows there’s multiple ways to navigate through the requests. Each API has this resource tree and it provides links to all of the relevant specification files for that API too, which is what makes it discoverable from within the browser. The next is an example of one of these resource specific files. These files contain everything we believe the developer needs to be aware of to perform that ”action”. For example it contains all the available http methods, the request and response schemas, all the parameters and requirements and also a set of examples, both request and response. This could be happy path or could involve an error scenario. Most importantly though, they highlight the rels or ”actions” that can be performed as a result of the original action. We really want our developers to think in terms of actions not endpoints.
• #10: Our API ref (which you can see a little example of is entirely generated from the specification files I just showed you. The contents ‘’tree’ on the left is comprised of all the individual API resource trees. We define some yaml config on our developer portal for each of these and the UI generate the relevant links. The link relationships you can see are what I mentioned previously and allow the user to discover and click through the docs for the available next actions. The right hand pane is used to display the code from our examples. By selecting an example from the middle this will populate the right hand side and allow you to run through some scenarios.
• #11: Our main API guide is very much focused around the concepts and use cases. We’re aware that integration of hypermedia driven APIs is done a little differently and so we want to use the guide to try and educate and ensure the integrator follows the right principles and best practices and also to help our developers understand why they are performing actions at certain stages. For us one of the key principles to get across is to avoid hardcoding the URLs. As I mentioned before the only URLs you need to be aware of are the API root resources so want to make sure our docs make that clear. We’d like to start fleshing the guide out a lot more as the product matures. We want to make it easier to drop in and grab a code snippet or have short tutorials to help somebody achieve a quick win.
• #12: Beyond the documentation, we figured that now we have these specifications we should put them to good use. We can’t make use of the great ecosystem that OpenAPI has so lets focus on something that will help visualize our APIs and the notion that each of them acts very much like a tree. We use our resource trees to generate the above diagram. As you can see, we have a handful of scenarios and the tree updates to show the flow of requests to achieve that. Going back to our refund example, this is one of the final actions and only performed once 3 previous requests have been made.
• #13: So where do we go next. We have our resourceTrees and our specs so we want to use them to generate more interactive assets. We have our API reference but we need to make it more interactive and so we’re working on bringing sandbox functionality. And finally we want to look at how we can begin to open-source some of what we’ve done. Making our spec available for everyone, this probably a fair way off at the moment however.
• #14: To recap, HATEOAS is definitely not for everyone but it works well for us. Discoverability can help ease some of the pain and burden on the docs When it comes to hypermedia driven APIs it’s important to educate and guide people to the best practices And yeah it’s all about trees, we love them.
• #15: Thanks you so much for listening.