Simple NGINX log file analyzer

Question

I'm looking for a simple tool to analyze my NGINX logs on my macbook. I want to be able to get some basic stats including bot access and also be able to sort/filter the logs to find out what happened right before the server crashed.

score 132 · Answer 1 · edited Jun 18 '20 at 15:37

132

You may try with GoAccess; free and open source console based. It may output an HTML report too.

edited Jun 18 '20 at 15:37

dusan

9,104
3
35
55

answered Jan 15 '13 at 16:04

Alex Mazzariol

2,456
1
19
21

18

Thanks for pointing out goaccess - simple to install (apt-get install goaccess on Debian/Ubuntu) and does quite a lot of nice stuff right on the command line right out of the box. No setup. – rfay Jun 30 '13 at 21:42
7

Also installs with `brew install goaccess` on Mac. – Matt Oct 06 '14 at 21:02
To install on Mac OS X without relying on homebrew, check the goaccess faq [here](https://goaccess.io/faq), specifically the section _How to install GoAccess from source under OS X El Capitan?_ – Anupam Aug 03 '17 at 05:40
4

with the default NGINX combined format, `goaccess --log-format=COMBINED access.log` or `{ cat access.log; zcat access.log.*.gz; } | goaccess --log-format=COMBINED -` (for all logs) worked well for me. – Sean Breckenridge Apr 28 '20 at 03:04

score 30 · Answer 2 · edited Oct 24 '22 at 20:10

30

I've come across visitors. It's written in C and is pretty fast.

It lacks GoAccess' cool ncurses interface. It only outputs html. It does the job, though.

It can also output human-readable text reports:

visitors access.log | less

edited Oct 24 '22 at 20:10

snwflk

3,341
4
25
37

answered Aug 28 '13 at 17:19

moebius_eye

574
6
14

score 7 · Answer 3 · answered Jul 07 '14 at 10:23

7

I will suggest to use logstash and Kibana with following patterns :

for access.log :

NGINXACCESS %{IPORHOST:clientip} - %{USERNAME:remote_user} \[%{HTTPDATE:time_local}\] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:http_user_agent}

for error.log:

 NGINXERROR (?<timestamp>%{YEAR}[./-]%{MONTHNUM}[./-]%{MONTHDAY}[- ]%{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}: %{GREEDYDATA:errormessage}(?:, client: (?<clientip>%{IP}|%{HOSTNAME}))(?:, server: %{IPORHOST:server})(?:, request: %{QS:request})?(?:, host: %{QS:host})?(?:, referrer: \"%{URI:referrer})?

answered Jul 07 '14 at 10:23

David

906
1
12
15

elk stack is the most powerfull solution if well configured. – thclpr Apr 02 '15 at 12:54
but requires a lot of GB and some basic VPS cannot simple run it – realtebo Mar 08 '19 at 14:07
Agreed with you – David Mar 30 '19 at 23:44

score 6 · Answer 4 · answered Dec 01 '12 at 12:21

6

The request-log-analyzer gem now has Nginx support in the master branch and i'm looking for someone to test it. http://github.com/wvanbergen/request-log-analyzer.

answered Dec 01 '12 at 12:21

Bart ten Brinke

1,063
7
5

1

can it handle both the NGINX access logs and error logs? – uwe Dec 01 '12 at 16:52

score 3 · Answer 5 · answered Sep 27 '12 at 21:31

3

I've now tried Splunk, it's a pretty powerful tool and they have a free version as well.

answered Sep 27 '12 at 21:31

uwe

3,938
11
37
50

Wow, it's pretty great! The free version limits you 500MB of new logs per day, which is generous enough. – chichilatte Mar 03 '15 at 22:37
1

I don't know the product yet, but the website feels very confusing to me – Neithan Max Dec 04 '19 at 09:52

Simple NGINX log file analyzer

5 Answers5

Related