Securing a HTTP API

Question

I need to secure a public facing HTTP API where I can not touch the code on the API server.

The HTTP API has multiple end-users that will consume it using non-interactive clients (typically backend services). Just to be clear, the client owns the resources that it will access and as such must provide a user since authorisation logic needs to be tied to a end-user.

I’m toying with the idea of using OAuth2 and the Resource Owner Password Credentials Grant and then using the access token provided to get a JWT which the client can present to a HTTP proxy that parses the request before passing it to the HTTP API Server.

Here is the flow as I envision it:

 +----------+                                  +---------------+
 |          |>--(A)---- Resource Owner ------->|               |
 |          |         Password Credentials     | Authorization |
 | Client   |                                  |     Server    |
 |          |<--(B)---- Access Token ---------<|               |
 |          |    (w/Refresh Token)             |---------------|
 |          |                                  |               |
 |          |>—-(C)---- Request JWT ——-------->| JWT Service   |
 |          |         (w/Access Token)         |               |
 |          |                                  |               |
 |          |<--(D)---- JWT ------------------<|               |
 |          |                                  |               |
 +----------+                                  +---------------+
       v
       |
       |
       |                                       +---------------+
       |                                       |               |
       |                                       |     HTTP      |
       --(E)---- HTTP Request w/JWT ---------->|     Proxy     |
                                               |               |
                                               |      (F)      | 
                                               |               |
                                               +---------------+
                                                       v
                                                       |
                                                      (G)
                                                       |
                                                       v 
                                               +---------------+
                                               |               |
                                               |     HTTP      |
                                               |      API      |
                                               |               |
                                               +---------------+
 
 (A), (B), (C) Get an access token using the Password Grant flow.
 (D) Use access token to get a JWT.
 (E) Attach JWT to HTTP request and send it to the HTTP Proxy.
 (F) Check that JWT is valid.
 (G) Pass request to the HTTP API Server.

Has anyone else solved a similar use case and would care to shed some light or have a discussion?

Are you married to the idea of going with Oauth? Oauth is great if you're going for a big distributed setup, but for a "secure this 1 api", any kind of header-based authentication will realistically do, and be super-simpler to implement — bbozo, Oct 05 '17 at 09:08
I'm not committed to anything at the moment but I do like the idea of short lived access tokens that can be revoked and I seem to get that out-of-the-box with Oauth2. — BusterX, Oct 05 '17 at 09:12
Or, in this particular use case it will be the JWT that is short lived and the client will need to support refreshing the JWT using it's Oauth access token or refresh token. If that fails it can just authenticate using the Password grant. — BusterX, Oct 05 '17 at 09:17
There's nothing "out of the box" about oauth 2 -_- https://hueniverse.com/on-leaving-oauth-f8dadb46d93f https://hueniverse.com/oauth-2-0-and-the-road-to-hell-8eec45921529 "In fact, one of the compromises was to rename it from a protocol to a framework, and another to add a disclaimer that warns that the specification is unlike to produce interoperable implementations." — bbozo, Oct 05 '17 at 13:07
What is the JWT Service supposed to do? Why do you want to send your custom JWT to the API instead of the access token? — Ján Halaša, Oct 05 '17 at 16:23
>There's nothing "out of the box" about oauth 2 "Out of the box" in the sense that a spec compliant OAuth 2 Server will support access and refresh tokens for specific claims. I'd rather not roll this kind of functionality my self and get superhacked. :) — BusterX, Oct 05 '17 at 17:35
@JánHalaša The JWT would be used to pass information related to authorization that the HTTP frontend would parse in order to determine if the user has access or not. I suppose I could use the Oauth 2 Scopes and the original Oauth 2 access token to accomplish the same. — BusterX, Oct 05 '17 at 17:47

score 1 · Answer 1 · answered Oct 05 '17 at 15:31

OAuth2 has a number of advantages ...

It has a clear flow and multiple types of grants it can use to cater to different needs.

Another advantage is that there are libraries that deal with the complexities of OAuth2, such as Identity Server : https://identityserver.github.io/Documentation/

Whether it is overkill for your project or not, only you can answer that question. A lot of people who claim OAuth2 is complicated haven't really spent enough time trying to understand it.

What I advise you not to is to not rely on any kind of self baked security model as this is what causes the downfall of a system. OAuth2 libraries have been battle tested by lots of users and companies.

A lot of companies which provide apis do so via OAuth2.

So, bottom line, if you do want to use it, do your research ,understand it and then implement it.

As for your actual question, yes I have built similar systems with lots of users, using various grants and everything worked quite well. There's nothing to be scared about as long as you spend enough time knowing what you get yourself into ...

I agree, research is a good thing. This thread is helping. Right now it is not clicking for me how to use OAuth 2 exclusively for this particular use case. I need to **support non-interactive clients**. That leaves me with only being able to use the [Resource Owner Password Credentials Grant](https://tools.ietf.org/html/rfc6749#section-4.3). That's fine for backend services that can keep a secret but what if I wanted to scale this to support user-agent based clients? — BusterX, Oct 06 '17 at 06:23
Also, think of the implications of maintaining a non-interactive client. You have a single bearer token across the entire cluster, so you will need a key-value store or a DB table to hold it. You will need to capture specific errors that mean the bearer has expired, in which case you will want to seamlessly request the bearer again without losing requests. Problem here is that this on a busy site can start to happen in parallel on 100s of threads. So, you need a distributed locking mechanism to do it right - redis mutex is my poison of choice. That, or just SHA512 a secret + nonce + http body — bbozo, Oct 06 '17 at 08:40

bbozo · Answer 2 · 2017-10-06T10:25:11.943

I'm getting downvoted for this answer, so I better explain myself.

I'm not the one to suggest "just write your own security libs", but I do make an exception with oauth+api clients (and especially oauth2).

Why not Oauth2?

extra hops & extra system components as compared to a traditional authentication scheme,
chances are that whatever you do, someone using some other programming language might not have a client library compatible with whatever you're using, people make money making oauth interoperable and simple
- think about this: nobody makes money making ie basic authentication "simple, compatible with 1000s of providers and just working" (to quote oauth.io), that's because basic authentication just works on every "provider" and oauth2 is a bit of a shitty, complex, non-interoprable framework - that's why we call basic authentication a part of a "protocol", we call oauth(1) a protocol, but we call oauth2 a framework
think of the implications of maintaining a non-interactive client:
1. you have a single bearer token across the entire cluster,
2. so you will need a distributed key-value store or a DB table to hold it
3. You will need to capture specific errors that mean the bearer has expired,
4. in which case you will want to seamlessly request the bearer again and retry the request (without losing requests).
5. Problem here is that this on a busy site can start to happen in parallel on 100s of threads
6. So, you need a distributed locking mechanism to do it right - redis mutex is my poison of choice when someone greets me with an oauth api
7. That + good luck testing that piece of complex distributed race condition logic, and when you break your back to do it (hi webmock) and then you still get random non-deterministic failures from time to time because the gods of concurrency met some combination of conditions that VCR/webmock didn't handle nicely
this, or just SHA512 a secret together with a nonce and HTTP body

According to the lead author of the Oauth2 project: (my emphasis)

All the hard fought compromises on the mailing list, in meetings, in special design committees, and in back channels resulted in a specification that fails to deliver its two main goals — security and interoperability. In fact, one of the compromises was to rename it from a protocol to a framework, and another to add a disclaimer that warns that the specification is unlike to produce interoperable implementations.

When compared with OAuth 1.0, the 2.0 specification is more complex, less interoperable, less useful, more incomplete, and most importantly, less secure.

To be clear, OAuth 2.0 at the hand of a developer with deep understanding of web security will likely result is a secure implementation. However, at the hands of most developers — as has been the experience from the past two years — 2.0 is likely to produce insecure implementations.

...

In the real world, Facebook is still running on draft 12 from a year and a half ago, with absolutely no reason to update their implementation. After all, an updated 2.0 client written to work with Facebook’s implementation is unlikely to be useful with any other provider and vice-versa. OAuth 2.0 offers little to none code re-usability.

What 2.0 offers is a blueprint for an authorization protocol. As defined, it is largely useless and must be profiles into a working solution — and that is the enterprise way. The WS-* way. 2.0 provides a whole new frontier to sell consulting services and integration solutions.

What to do instead?

Oauth is typically overkill unless you're creating a bigger eco-system.

Simpler solution is DIY, define a custom authorization header as:

authentication_id api_client_id nonce digest

ie

FooApp-SHA512-fixed 4iuz43i43uz chc42n8chn823 fshi4z73h438f4h34h348h3f4834h7384

where:

authentication_id is a a fixed string that describes what kind of authentication is being used,
api_client_id is a public piece of information identifying the API client (I assume the API has more than 1 client, or that it will have more than 1 client at some point) - API client ID is there to allow you to match the API client with the API clients' secret
nonce is just a random string
secret is a random string known only to you and the client and the client should treat it as a password (ie not commit it to versioning)
digest is a SHA512 hex/base64 digest of the api_client_id + nonce + secret (you can add also concatenate the HTTP body, I'd add the HTTP body unless the body is big - such as with file uploads)

If the client passes the authentication simply forward the request to the backend API service and return it's response to the client, otherwise render an error.

I don't see why one would write their own custom protocol/format code and roll your own crypto whilst there are standard components to handle OAuth 2.0. See e.g. https://github.com/pingidentity/mod_auth_openidc for the proxy component. — Hans Z., Oct 05 '17 at 17:15
The above also provides MAC, it's simple to understand and reason about, requires less hops and system components to maintain, learning curve is simpler because securing an API isn't rocket science, we've been doing it for a long time now, most of "us" know how to do it by now without oauth2. Compare work-hours between setting up an oauth2 service, and integrating it, and just doing "what you usually do when you write an API" — bbozo, Oct 05 '17 at 21:13
And security-wise, you're probably more likely to shoot yourself in the foot by fiddling around with an oauth2 service provider you don't have experience with than writing an api authenticator as you normally would. And good luck then to set up mac checks if that's a thing for you. — bbozo, Oct 05 '17 at 21:18
@HansZ. I extended my answer to include the answer to "why one would write their own custom protocol/format code and roll your own crypto whilst there are standard components to handle OAuth 2.0" — bbozo, Oct 06 '17 at 09:02

Securing a HTTP API

2 Answers2

Why not Oauth2?

What to do instead?