Using Jsawk to parse JSON access logs

Question

With our new webservers, the access logs are in JSON and I'm not able to use typical awk commands to pull out traffic info. I've found jsawk, however I keep getting a parse error anytime I try to pull anything out of the access logs. I have the feeling that the logs are not in a format the the parser likes

Here is a sample entry from the logs:

{ "@timestamp": "2014-09-30T21:33:56+00:00", "webserver_remote_addr": "24.4.209.153", "webserver_remote_user": "-", "webserver_body_bytes_sent": 193, "webserver_request_time": 0.000, "webserver_status": "404", "webserver_request": "GET /favicon.ico HTTP/1.1", "webserver_request_method": "GET", "webserver_http_referrer": "-", "webserver_http_user_agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" }

So for example if I want to pull the IP addresses out of the logs, I would use this:

cat access.log | jsawk 'return this.webserver_remote_addr'

However this only results in 'jsawk: JSON parse error:' and the entire access log printed.

Am I correct in assuming that the access logs are in a format the parser doesn't recognize? Each entry in the logs is all on one line. How can I get jsawk to parse properly?

Answer 1

I tried this:

$ echo '{ "@timestamp": "2014-09-30T21:33:56+00:00", "webserver_remote_addr": "24.4.209.153", "webserver_remote_user": "-", "webserver_body_bytes_sent": 193, "webserver_request_time": 0.000, "webserver_status": "404", "webserver_request": "GET /favicon.ico HTTP/1.1", "webserver_request_method": "GET", "webserver_http_referrer": "-", "webserver_http_user_agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" }' | jsawk 'return this.webserver_remote_addr'

and got this:

24.4.209.153

Updates: I think the problem is that you have each line as a json object, and there are multiple lines in access.log. There's a good way to work around at here: How to use jsawk if every line is a json object ?