Fluid Web Services

An approach to building web services that allows the services to continually evolve with graceful fallbacks.

Manifesto

A fluid web service is one that will evolve as needed to be the best representation of the service provided. A consequence of this approach is that the service can change at any moment. To allow for this evolution controls are put in place that allow clients to gracefully fallback to previous versions of the service for a pre-agreed period of time while they upgrade to the current version.

Approach

Fluid web services use header flags to control graceful fallback and logging.

Fluid-From: A string that identifies the calling system or person.

Fluid-Fallback: An fallback URL returned from the server during the upgrade period that serves the previous web service version.

Fluid-Expiry: The number of days before the fallback endpoint will expire. Optionally followed by a message indicating the reason the endpoint is deprecated.

Getting Started

Consider a web service I designed that serves users. As this is a fluid web service the caller, Joe, includes a Fluid-From header. In the first version name and location are mandatory, hobby is optional. Joe cares about name and location and puts in checks in his code to ensure they are present.

In the examples ‘>’ indicates headers sent to server, ‘<’ indicates headers returned from server, other content is the payload.

> GET /users/Simon HTTP/1.1
> Fluid-From: Joe
< HTTP/1.1 200 OK
< Content-Type: application/json
{
  "name": "Simon",
  "location": "London",
  "hobby": "coding"
} 

Over time I realise the error of my ways: location is too general and should have been called city, also some people have more than one hobby. I roll out a new version of the web service that corrects these problems. I don’t need to let Joe know as this is a fluid web service (though I did cos I’m good like that). This is what happens when Joe next calls the service.

> GET /users/Simon HTTP/1.1
> Fluid-From: Joe
< HTTP/1.1 200 OK
< Content-Type: application/json
< Fluid-Fallback: /users-88af9d35/Simon
{
  "name": "Simon",
  "city": "London",
  "hobbies": [ "coding" ]
}

When Joe, who knows this was a fluid web service, sees the expected location field is missing he makes a new web service request to the fallback URL given in the Fluid-Fallback header.

> GET /users-88af9d35/Simon HTTP/1.1
> Fluid-From: Joe
< HTTP/1.1 200 OK
< Content-Type: application/json
< Fluid-Expiry: 21 Use updated /users structure
{
  "name": "Simon",
  "location": "London",
  "hobby": "coding"
}

With this Joe gets the data he wants in the format he can process. He also has noted the expiry days and warning and added “update to latest /users” endpoint onto his todo list. On the server side the warning has been logged, along with Joe’s Fluid-From identifier, allowing me to see the uptake, or not, of the updated endpoint.

Notes

• The fallback URL returned may be ephemeral or change on server restart so should not be cached by Joe.
• The expiry period (the number of days a fallback URL will be published following service update) will be published by the service provider along with the other web service documentation.
• Other uses of the web service that don’t care about reading location or hobbies continue to work and don’t need to use the fallback URL, or change any code.

Alternatives

There are two main alternatives, both of which while workable are not perfect and do not fit the manifesto.

Explicit Endpoint Versioning

With explicit endpoint versioning the version of the web service is encoded into the URL so for example the first version of users would have the url

> GET /users/v1/Simon HTTP/1.1

When the updated version is released a new endpoint is created

> GET /users/v2/Simon HTTP/1.1

The main drawback of this approach is that the web service provider is stuck with multiple versions of the endpoint for the long term. This can quickly get complex and confusing for both maintainers and clients of the API.

Field Duplication

With field duplication both new and old field names are included in the structure.

{
  "name": "Simon",
  "city": "London",
  "location": "London",
  "hobbies": [ "coding" ],
  "hobby": "coding"
}

So a messy structure backed by messy documentation is produced. It is not obvious to new users of the service if they should be using city or location. As we don’t know which fields are being used the service provider has to support these duplications for the long term.

Wrap Up

We have discussed how to get started with Fluid Web Services and shown how they can keep a web services API current and clean. There is a lot more to discuss including:

• How changes to optional fields (e.g. hobby/hobbies) are detected
• How to evolve payloads sent to the server from the client
• Tooling to help with compliance reporting and fallback endpoint generation.

To find out more about these topics please comment or get in touch.