7

I'm building a RESTful service in Symfony2, using the FOSRestBundle. I can track page usage in web clients using Google Analytics. However, this is is obviously not going to work for requests by non-HTML clients.

Before I embark on installing Redis, writing services, event dispatchers, etc, has this problem already been solved? Is there a solution that doesn't have a serious impact on performance?

Based on stats of the project I'm replacing, I expect around 1,000 hits per hour with 90% of traffic coming from browsers. I won't be in control of the non-HTML clients, so adding tracking there is not an option.

I need the data for the same reason that anybody needs analytics data - to make pretty graphs, and give quantitative evidence about where to focus development resources.

k0pernikus
  • 60,309
  • 67
  • 216
  • 347
Dan Blows
  • 20,846
  • 10
  • 65
  • 96
  • 4
    Maybe you could search for something that analyses your access log (no impact on performance at all) – greg0ire Aug 30 '12 at 20:02
  • Not a bad idea. I don't know much about Apache - how can I tell it to log the `Accept` header for every request? – Dan Blows Aug 30 '12 at 20:22
  • I know you can customize the log format. See the [LogFormat](http://httpd.apache.org/docs/2.2/mod/mod_log_config.html#logformat) directive documentation. – greg0ire Aug 30 '12 at 20:25
  • There you go! http://stackoverflow.com/questions/8464863/is-there-a-google-analytics-like-solution-to-track-rest-calls-to-a-web-service – Max Małecki Aug 31 '12 at 08:05
  • Great question. I have the same problem as you. I'm very keen to see who can suggest the cleanest solution. – Flukey Aug 31 '12 at 10:17
  • Is your API will be used by specific device or is it a public api? If not there several SDK to allow tracking on Mobile Device for example. – Boris Guéry Aug 31 '12 at 14:01
  • @BorisGuéry the API will be public, to those with registered accounts, so I don't think we could do that. – Dan Blows Aug 31 '12 at 14:54
  • The idea of analyzing the log files is a good idea but won't allow a fine grained tracking analysis. I actually don't know about any "pluggable" solution, and I don't think they will ever be one since it is really application-specific. You'd rather go with the EventDispatcher and custom Events coupled with a custom Logger. It is **really** not much work to do. Because you're likely to use Event the proper way first to extend your application, and just inject a logger to audit actions. – Boris Guéry Aug 31 '12 at 15:32

1 Answers1

2

There are a few options, and it depends on exactly how much information you need. If you just want to know basic activity (how many calls to each web service, when, source IP, user agent), then the apache logs already have all that information. Use CustomLog to add any additional fields you need. E.g. you mentioned the Accept header, which can be added like this:

CustomLog logs/access_log "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" \"%{Accept}i\""

If it is GET, POST, PUT, etc. is contained in the %r part.

If you want to know what is actually in the POST or PUT data, then it is more difficult. The extreme solution is to use the mod_dumpio module. That logs all of the client input (all headers, all cookies, all POST data). If people use your REST API to upload images then, for good or bad, you get the full image in your log. That could get very big.

The solution I favour is to log from PHP: a custom log either at the top of your PHP script, or when you process the requests. You then have full control over what to log, the easiest format to analyze, and you can also put it in context (e.g. log text data, but not logging image bytes). In development and for small sites I do this in parallel with apache logging. If Apache is taking too much CPU, then disable apache logging, or bypass Apache completely. (I'm currently evaluating the built-in webserver in php 5.4 - it has support for routing, so could be very suitable for web services.)

BTW, analyzing server logs is good to do in parallel with Google Analytics: it helps you appreciate the accuracy of each.

Darren Cook
  • 27,837
  • 13
  • 117
  • 217