27

I am investigating options to build a system to provide "Entity Access Control" across a microservices based architecture to restrict access to certain data based on the requesting user. A full Role Based Access Control (RBAC) system has already been implemented to restrict certain actions (based on API endpoints), however nothing has been implemented to restrict those actions against one data entity over another. Hence a desire for an Attribute Based Access Control (ABAC) system.

Given the requirements of the system to be fit-for-purpose and my own priorities to follow best practices for implementations of security logic to remain in a single location I devised to creation of an externalised "Entity Access Control" API.

The end result of my design was something similar to the following image I have seen floating around (I think from axiomatics.com)

EAC API implementation concept

The problem is that the whole thing falls over the moment you start talking about an API that responds with a list of results.

Eg. A /api/customers endpoint on a Customers API that takes in parameters such as a query filter, sort, order, and limit/offset values to facilitate pagination, and returns a list of customers to a front end. How do you then also provide ABAC on each of these entities in a microservices landscape?

Terrible solutions to the above problem tested so far:

  • Get the first page of results, send all of those to the EAC API, get the responses, drop the ones that are rejected from the response, get more customers from the DB, check those... and repeat until either you get a page of results or run out of customers in the DB. Tested that for 14,000 records (which is absolutely within reason in my situation) would take 30 seconds to get an API response for someone who had zero permission to view any customers.
  • On every request to the all customers endpoint, a request would be sent to the EAC API for every customer available to the original requesting user. Tested that for 14,000 records the response payload would be over half a megabyte for someone who had permission to view all customers. I could split it into multiple requests, but then you are just balancing payload size with request spam and the performance penalty doesn't go anywhere.
  • Give up on the ability to view multiple records in a list. This totally breaks the APIs use for customer needs.
  • Store all the data and logic required to perform the ABAC controls in each API. This is fraught with danger and basically guaranteed to fail in a way that is beyond my risk appetite considering the domain I am working within.

Note: I tested with 14,000 records just because its a benchmark of our current state of data. It is entirely feasible that a single API could serve 100,000 or 1m records, so anything that involves iterating over the whole data set or transferring the whole data set over the wire is entirely unsustainable.

So, here lies the question... How do you implement an externalised ABAC system in a microservices architecture (as per the diagram) whilst also being able to service requests that respond with multiple entities with a query filter, sort, order, and limit/offset values to facilitate pagination.

Gary MacPherson
  • 512
  • 6
  • 15
  • 1
    So there are a couple of things here. If you're simply talking about access levels to entities by type (e.g. Mary can see `sales` and `leads` but Stewart can only see `leads`) then all you need is for EAC API to return mapping of entity types to permitted access levels, and then have your API filter the data without sending it over. If you want per-record (sharing) model where Mary can access `lead1` and `lead2` but Stewart can only access `lead1` then you need to store that information somewhere, and then it's a question of `do we have more rules than records` – zaitsman Feb 18 '20 at 05:46
  • 1
    If you have more rules, you'd send the records over. If you have more records, you'll pull the rules and then filter them out. – zaitsman Feb 18 '20 at 05:47
  • My use-case is definitely a "If you want per-record (sharing) model where Mary can access lead1 and lead2 but Stewart can only access lead1" situation. Not sure if I am following the resulting suggestion though. – Gary MacPherson Feb 18 '20 at 05:49
  • If the number of records is very high, then provide the rules to the API... but what if the rule is eg. "This requesting user can only see customers owned by sales Team X". That would mean that my customer API now needs to keep a record of which sales team owns them. While that isn't infeasible, it gets harder with more complex rules and It means that anything I use to determine the success/failure of the ABAC controls needs to be stored in each API, with the filtering logic now in each API. This is now no longer externalising the controls and each API is responsible for its own implementation – Gary MacPherson Feb 18 '20 at 05:54
  • Sorry, obviously the rules have to be contextual to the API you are accessing. So e.g. if you have one API for 'Customer' another for 'Sales Team-Customer' link and another for EAC then you have no choice but to limit the page size in your query (e.g. no more than 50 customers) and then have EAC do the leg work. Think in a monolith you would join 3 db tables for this. In microservices, you need to have these three tables collected across three APIs. If these don't change all too often you may dump them into some memory cache and you can optimise rules by groups – zaitsman Feb 18 '20 at 06:37
  • But ultimately this is now the business logic of your app, regardless of microservices the query for this data will slow down as any rules are evaluated on a per-record basis. Another caching suggestion that might work is if you have a limited number of users -> pre-create these payloads for them and flush if new records are created or new rules are added. – zaitsman Feb 18 '20 at 06:40

1 Answers1

9

After dozens of hours of research, it was decided that this is an entirely unsolvable problem and is simply a side effect of microservices (and more importantly, segregated entity storage).

If you want the benefits of a maintainable (as in single piece of externalised infrastructure) entity level attribute access control system, a monolithic approach to entity storage is required. You cannot simultaneously reap the benefits of microservices.

Gary MacPherson
  • 512
  • 6
  • 15
  • 1
    Segregated storage does not mean that you can't replicate. Users and entitlements (filtered even to the target type) can come from the identity/auth service, and be replicated in to a dozen services, as long as the auth service can represent those entitlements in a way that can decorate a query for each storage type (e.g. xSQL, Elasticsearch) to include the constraints. – Doug Moscrop Apr 10 '20 at 02:41
  • I agree with you, however the replication of user data/rules across all APIs means each API (and therefore the teams building those APIs) is now responsible for ensuring that the rules are all being enforced correctly. This harms the intent I was going for above "to follow best practices for implementations of security logic to remain in a single location". – Gary MacPherson Apr 14 '20 at 04:32
  • You can extend that best practice framing to the whole system, not just things like referential integrity, but all business logic. The word "maintainable" can mean different things to different people, as in, maintaining that central piece may be onerous in terms of scaling and operations and insulating features/functionality from each other in terms of fault isolation and what have you.. its not that I don't agree with you, but consider maybe entity storage "as a service" where one team manages a way to spin up storage and enforce those rules. Databases are just abstractions. – Doug Moscrop Apr 15 '20 at 05:33
  • Or even via a service mesh, where the databases for each service (which themselves are '"services") are isolated except through a driver/API that always amends the necessary statements to constrain the query, and one team maintains that. Can someone circumvent this? Well sure, in the same they can do a lot of harmful things - some of that is an organizational problem that isn't best solved with software but process (code review, other guards against manipulating or provisioning infrastructure, but I prefer to work in a more trusted environment)... – Doug Moscrop Apr 15 '20 at 05:42
  • Anyway, sorry for the spam, but if it's important enough of a thing for you then maybe consider architecting in a query abstraction layer to the system, perhaps via some pattern you fancy (DDD/repository aggregate, etc.) and give services a consistent way to request/provision storage and interact with it. – Doug Moscrop Apr 15 '20 at 05:44
  • 1
    I think this question and discussion do an outstanding job of directly addressing a problem I have struggled to think through in ABAC. I like the proposal that "the auth service can represent those entitlements in a way that can decorate a query for each storage type (e.g. xSQL, Elasticsearch) to include the constraints." @DougMoscrop 's suggestion that it is possible to accomplish that by a design pattern makes sense, but also feels like too basic a place to start. Surely there are better authorization frameworks available now? – James Jul 12 '20 at 13:30
  • Differently put: the translation from entitlements to query decoration *is* a policy. Axiomatics, mentioned above, seems to have several products aimed at just this problem (especially their "dynamic authorization"). Are there other policy languages/policy decision point implementations/authorization services that we could consider to use to provide similar functionality? – James Jul 12 '20 at 14:04
  • Have anyone looked at [OPA](https://www.openpolicyagent.org/)? It's possible to see it in action with a demo in this [meetup](https://www.youtube.com/watch?v=73buTTjYITk). I'm currently researching about ABAC in microservices but it's obviously that you guys have more experience in this topic. – Asdrubal Sep 17 '20 at 21:54
  • OPA is an implementation of ABAC and the externalized authorization pattern. As such, it has exactly the problems mentioned by Gary – Daniel Hilgarth Oct 01 '20 at 07:19
  • Have you read this article? "Implementing ABAC in a Microservice Architecture" https://www.identityserver.com/articles/implementing-abac-in-a-microservice-based-architecture – luenib Sep 21 '21 at 14:32