Full record url in nginx log

Question

We use following nginx site configure file in our production env.

log_format main '$http_x_forwarded_for - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" $request_time';



server {
        root /srv/www/web;
        server_name *.test.com;
        access_log /var/log/nginx/xxx.test.com.access.log main;

Both "http://a.test.com/ping" and "http://b.test.com/ping" http request will be record in file xxx.test.com.access.log.

But there is a problem, nginx don't store "domain name" in xxx.test.com.access.log.

"http://a.test.com/ping" and "http://b.test.com/ping" share the same request "Get /ping".

How can I record "a.test.com" or "b.test.com" in nginx log?

score 78 · Accepted Answer · edited Apr 14 '16 at 14:38

78

Try adding the $host variable in log_format:

log_format main '$http_x_forwarded_for - $remote_user [$time_local] "$host" "$request" '
            '$status $body_bytes_sent "$http_referer" '
            '"$http_user_agent" $request_time';

http://wiki.nginx.org/HttpCoreModule#.24host:

$host

This variable is equal to line Host in the header of request or name of the server processing the request if the Host header is not available.

This variable may have a different value from $http_host in such cases: 1) when the Host input header is absent or has an empty value, $host equals to the value of server_name directive; 2) when the value of Host contains port number, $host doesn't include that port number. $host's value is always lowercase since 0.8.17.

edited Apr 14 '16 at 14:38

robsch

9,358
9
63
104

answered Jan 15 '14 at 13:44

Tan Hong Tat

6,671
2
27
25

We use "server_name *.test.com;" in nginx.conf, so "$host" is "*.test.com". – user1859451 Jan 16 '14 at 03:31
It works for me on Nginx 1.4.4. Have you "reload" nginx? – Tan Hong Tat Jan 16 '14 at 10:52
2

This is the predefined combined format (with $host added), for anyone who needs it; the questioner's example doesn't include `$remote_addr`, which many people will want for a standard user's IP address: `log_format combined '$remote_addr - $remote_user [$time_local] ' '"$host" "$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent"';` Source: http://nginx.org/en/docs/http/ngx_http_log_module.html – William Turrell Apr 03 '18 at 12:13
@TanHongTat only for the first request hostname is get logged and simultaneously requests won't be logged. Is there a way to log simultaneous requests too – Mahesh.D May 31 '19 at 14:17

Tim Bray · Answer 2 · 2019-04-11T08:14:23.407

75

If you want to log the full requested url, then my method is

log_format main '$http_x_forwarded_for - $remote_user [$time_local] '
'"$request_method $scheme://$host$request_uri $server_protocol" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" $request_time';

So splitting the $request into its constituent parts and popping $host in the middle. Also lets you see if the request is http or https.

edited Apr 11 '19 at 08:14

answered Jun 17 '16 at 08:57

Tim Bray

1,373
10
7

12

This format has the advantage of being parseable by things that expect the standard format, like the filebeat nginx module and the grok patterns for logstash – kbolino May 28 '18 at 17:52
3

Thanks. In my case, first field is better with `$remote_addr` – Fernando Fabreti Apr 09 '19 at 19:56
2

`$http_x_forwarded_for` helps you to capture client IP in multiple cases such as if the client is connecting directly as well as client is connecting via a proxy such as Cloudflare. – Ashutosh Kumar Sep 02 '21 at 07:57

Full record url in nginx log

2 Answers2