4

Contemplating building an Angular 2 front-end to my website. My question is not necessarily related to Angular but I want to provide full context.

Application logic that displays content to user would shift to the client. So on the server side, I would need to expose data via a RESTful JSON feed. What worries me, is that someone can completely bypass my front-end and execute requests to the service with various parameters, effectively scraping my database. I realize some of this is possible by scraping HTML but exposing a service with nicely formatted data is just a no-brainer.

Is there a way to protect the RESTful service from this? In other words, is there a way to ensure such service would only respond to my Angular 2 application call? Authentication certainly isn't a solution here - I don't want to force visitors to authenticate and the scraper could very well authenticate and get access, anyway.

jacekn
  • 1,521
  • 5
  • 29
  • 50
  • Make it so your API requires a token (sent with every request) and you can validate this token server-side to allow/forbid certain actions. – Simon Arsenault May 13 '16 at 15:48
  • I found this article pretty useful: https://stormpath.com/blog/where-to-store-your-jwts-cookies-vs-html5-web-storage – Rob May 13 '16 at 15:56
  • 1
    Tokenization and throttling all the way, this makes API no more attractive than scraping. – Estus Flask May 13 '16 at 16:11
  • Scrapers are techies. They can authenticate, look at headers and code requests that do exactly the same thing from their own program. – jacekn May 13 '16 at 17:05

3 Answers3

1

There is no obvious way to do it that I know of, but a lot of people seem to be looking at Amazon S3 as a model. If you put credentials in your client code, then anyone getting the client code can see them. I might suggest that you could write the server to pass a time limited token back to the browser with the client code. The client code would be required to pass it back to the server for access. This would prevent anyone from writing their own client code, as only client code sent by the server would work, though only for some period of time. The user might occasionally get timeouts, but that depends on how strict you want to make the token timeouts. Of course, even this kind of thing could be hacked by someone making a client request to get a copy of the token to use with their own client API, but at that point you should be proud that someone is trying so hard to use your API! I have not tried to write such a thing, so I don't have any practical experience with the issue. I myself have wondered about it, but also don't have enough experience with this architecture to see what, if anything, others have been doing. What do angularJS forums suggest?

Additional References: Best Practices for securing a REST API / web service

Community
  • 1
  • 1
K.Nicholas
  • 10,956
  • 4
  • 46
  • 66
  • Nicholas, you already stated yourself that getting the next valid token programmatically would be doable. It may be work but then I have a database that requires a lot of time to maintain (data collection + entry). Spending a bit of time to hack a time limited token to get ready and beautifully formatted data, still saves the scraper a ton of time. – jacekn May 13 '16 at 17:10
  • 2
    The poster is really asking a digital copyright question. If you put your data on a public interface, what's to stop people from using it? You can make it more difficult, or require personal access credentials, but even then you need someone to go an look for copyright abuses and an attorney for cease and desist demands. Queue up the entire entertainment industry. – K.Nicholas May 13 '16 at 17:48
1

I would recommend JWT Authorization. One such implementation is OAuth. Basically you get a json web token ( JWT ) that has been signed by an authority you trust that tells about the user and what resources they can access on your api.

If the request doesn't include an Authorization token - your API rejects it.

If the token has been tampered with by someone trying to grant themselves privledges after the token is signed by the authorization authority - your API rejects it.

It is a pretty cool piece of kit.

This site has information about OAuth implementations in different languages, hopefully your favorite is listed.

Some light bed time reading.

Michael
  • 1,028
  • 18
  • 25
0

I believe the answer is "No".

You could do some security by obscurity type stuff. Your rest API could expose garbled data and you could have some function that was "hidden" in your code un-garble it. Though obviously this isn't fool proof, but if you expose data on a public site it's out there regardless of server or client rendering.

JoeB
  • 2,743
  • 6
  • 38
  • 51
  • 2
    No, just no. Don't do security through obscurity. – Simon Arsenault May 13 '16 at 15:51
  • Actually, I think Joe makes a good point here. Basically, the best solution thus far, would be to mess with the JSON feed so much that scraper's program would get continually confused. So, false records, meaningless tokens inserted here and there, maybe even invalid JSON in some ways... This could be done. It just doesn't feel like the right path... This will be a real challenge for these JS solutions that are gaining popularity. – jacekn May 13 '16 at 17:04
  • @jacekn If the one who does scraping for living will be confused with your API, imagine how your team will be. This may take extra couple of days to RE the stuff, how much will it cost to develop and maintain it? – Estus Flask May 13 '16 at 21:31
  • We could debate this point but it's not worth it. Basically, Joe stated there is no way to protect service calls, only to mess with the feed. No-one provided any ideas on real scraping protection, so I accept Joe's answer and I'm moving on without Angular. – jacekn May 15 '16 at 13:35