11

I have a Rails 3 engine which exposes API routes for around 20 controllers. Those controllers represent several different resources at various levels of nesting and are covered by over 500 rspec tests. The API is versioned at v1 using namespaces and a routing constraint based on a version header with a default to v1. This is the versioning system described in a great many blog posts and seems to be best practice.

What none of those blog posts describe is how you actually manage rolling out a new version. I have to make a breaking change to the output of a single controller. This change affects the JSON response of an object by changing the structure of one of the JSON values. This will cause breaks in the index, show, and edit views for that controller.

It's obvious that I can copy all of app/api/v1 to app/api/v2[1]. I can then make my single change to my new v2 serializer. I've now got a large amount of duplicated code for a version 2 of an API that has made almost no changes. I need to maintain code in two places. I'll probably have to have my whole rspec suite run on the version 2 controllers as well as the version 1 ones with a tiny bit of extra testing for the v2 serializer. This sounds like a horrible idea. We could have some stub v2 controllers for each unchanged controller in the v1 namespace that inherits from the v1 controller. This doesn't sound very nice either.

The best option that I can think of is to have a single controller (in this case probably just a single serializer) inside my v2 API, with some routing magic to check if a controller for the required version exists and to fall back through previous versions until it finds one. The serializer version should also have similar magic to check if one exists for this version and fall back until it finds one. This introduces minimal extra code and doesn't instantly double the duration of my test suite. It would require being able to plug in a function directly into the rails routing logic before it could return a 404 for my missing v2 controllers. Possibly I could analyse the namespaces for all controllers based on the filesystem and generate routes at rails boot time with fallbacks but it would be difficult to manage explicitly removing routes from a previous version of the API.

It seems that we will need to continue doing this for every non-additive functionality/output format change up to the point that each previous version is deprecated and removed. We have an additional unreleased API consisting of ~75 controllers covered by ~4000 specs. What happens when we start externally documenting and releasing these?

Other than batching up API changes which is not feasible at the rate that we release features, how do other people manage this? Is the idea above possible at all? Is there a better way?

[1] Issue one. We're using ActiveModel::Serializers for producing JSON responses. ActiveModel::Serializers doesn't support API versioning, although there seems to be a way around this using ruby magic to pick the correct class.

Martin Foot
  • 3,594
  • 3
  • 28
  • 35
  • Have you read the answers to this question? Seems pertinent. http://stackoverflow.com/questions/389169/best-practices-for-api-versioning – stolli Aug 21 '15 at 18:31
  • Martin, did you solve the problem based on the answers below? – Anatoly Sep 29 '15 at 23:03

2 Answers2

6

The project ActiveModel::Serializers has number issues related to Versioning, one of them provided an idea how to implement versioning via Namespace modules but it was closed 2 days ago followed by one of developer's words:

As you noticed we have discussed versioning on other issues and PR's as well, and I'm glad to read so really nice thought from all of you.

So the problem with AMS versioning does exist but not solved yet.

Back to the original question:

It's obvious that I can copy all of app/api/v1 to app/api/v2. I can then make my single change to my new v2 serializer. I've now got a large amount of duplicated code for a version 2 of an API that has made almost no changes. I need to maintain code in two places.

There is a compromise between inheritance complexity with side effects VS code duplication. In case of having well-tested V1 codebase that should be locked for any modification the maintenance does mean to have no errors while running regression test suite. The version 1 development cycle finished, tests written, contract behaviour signed off. The code duplication V1-V2 makes sense and it avoids regression failures.

I'll probably have to have my whole rspec suite run on the version 2 controllers as well as the version 1 ones with a tiny bit of extra testing for the v2 serializer. This sounds like a horrible idea.

I don't agree it's a horrible idea, this is a trade-off between expected behaviour and imaginary convenience with development. It's also not easy to avoid spec suite to be duplicated. Controllers, models can be reused but spec codebase will be more likely duplicated to be 100% confident that new changes don't break previous API version.

The best option that I can think of is to have a single controller (in this case probably just a single serializer) inside my v2 API, with some routing magic to check if a controller for the required version exists and to fall back through previous versions until it finds one.

Yes, this sounds good and helps to avoid application code (not spec suite though) duplication but requires an additional development efforts with maintenance. What you are trying to do called copy-on-write, only changes are copied over. This is well-known optimisation technique. Nevertheless the HTTP fallback sounds more appropriate.

Possibly I could analyse the namespaces for all controllers based on the filesystem and generate routes at rails boot time with fallbacks but it would be difficult to manage explicitly removing routes from a previous version of the API.

Imagine you have more than 2 versions of API, and a certain API call has 2 fallback ancestors where second is broken by developer's mistake, will you intercept not only 404 but 500 exceptions as well? What if latest DB scheme version breaks backward compatibility?

We have an additional unreleased API consisting of ~75 controllers covered by ~4000 specs. What happens when we start externally documenting and releasing these?

This is more like architecture question rather than specific implementation. If the API tends to be big, API design patterns can help to avoid building monolith API that can be difficult to support and maintain.

What would I recommend to do:

  • duplicate V1 to V2 entirely, rspec suite included
  • don't be afraid to spend 2x time on running tests
  • wait until AMS release a versioning (release v0.10.x)
  • split monolith API to the individual ones based

If code duplication is not acceptable the other option is to duplicate Rails app and deploy to the same server and dispatch requests with Nginx configuration:

location /v1 {
  proxy_pass http://http://unix:/tmp/v1_backend.socket:/v1/;
}

location /v2 {
  proxy_pass http://http://unix:/tmp/v2_backend.socket:/v2/;
}

This particular code shows just for an example, I don't say it's a good idea to have 10 different Rails apps with each own version.

Back to original question, API versioning is difficult and for some API-clients it makes sense to have default (latest) API URL endpoint.

Anatoly
  • 15,298
  • 5
  • 53
  • 77
  • 1
    +1 good answer, also I'd add that if you want to share helper functions between versions you can extract them into a mixin, and you can run tests against the mixin, and as soon as functions start to change in the common function you need to move that code outside of the mixin and into the controllers, also, use a form object to handle your apis, it'll be simpler to test against (it's much simpler to unit test a form class than a controller) https://www.reinteractive.net/posts/158-form-objects-in-rails – bbozo Aug 24 '15 at 08:27
  • All the fun begins when you migrate your database (and you usually do that often when going to next version of your API). How do you handle that case? – EugZol Aug 24 '15 at 20:03
  • @EugZol there is no other answer as to keep backward compatibility and drop it when deprecate older version of API. – Anatoly Aug 24 '15 at 20:04
  • Yes, and you can't really "sign off" old versions of API, they need to constantly adapt to your new schema as well. – EugZol Aug 24 '15 at 20:06
4

If I understood your all demands correctly, wouldn't this be sufficient solution for routing the v2 requests:

  1. Check for resource existence under v2.
  2. If not found, check that it isn't one of the disabled resources. If it is, return 404.
  3. Fallback for v1 resource if one is found.

Here's an example code (one scope for each step in the list above)

scope constraints: lambda { |request| request.url.split('api/')[1].split('/')[0] == 'v2' } do
  # New resources introduced in v2
end

# Resource was not found in v2 API, check if it is removed
scope constraints: lambda { |request| request.url.split('api/')[1].split('/')[0] == 'v2' } do
  # Resources removed from v2
  resources :resource1, to: proc { [404, {}, ['']] }
end

# Fallback for v2 routes that don't have v2 controller defined
scope constraints: lambda { |request| ['v1', 'v2'].include?(request.url.split('api/')[1].split('/')[0]) } do
  # Original v1 resources
end

About the serializers, as you mentioned yourself those can be easily fixed by always providing new controller when changing them, or even by doing some magic that checks the version from the URL in default_serializer_options and sets serializer based on that.

Roope Hakulinen
  • 7,326
  • 4
  • 43
  • 66