Severity level on docker logging to syslog

Question

I have a syslog-ng container that collects logs from other containers running on the same application, and normal logging works as intended: every container send its logs and syslog-ng saves them on separate files, as defined in syslog-ng.conf.

I'm also trying to save error logs on a different file. I was assuming that everything thrown to STDERR from inside another container should be seen by syslog-ng as an error, so when I started a container like this:

docker run -dit --log-driver syslog --log-opt syslog-address=tcp://my_syslog:601 alpine echo "boom" > /dev/stderr

then I expected syslog-ng to understand it to be an error, but it does not.

This is an extract of my syslog-ng.conf inside the syslog-ng container:

@version: 3.25
source s_net {
  network(transport(tcp) port(601));
  network(transport(udp) port(514));
};
destination d_error_file {
  file("/tmp/errors");
};
filter f_errors {
  level(err .. emerg);
};
log {
  source(s_net);
  filter(f_errors);
  destination(d_error_file);
};

How can I tell syslog-ng to treat STDERR messages as errors?

Oluwafemi Sule · Accepted Answer · 2020-11-09T05:32:42.570

My original reply to set up a /dev/stderr source was misinformed. I thought that syslog-ng was running in the host.

As syslog-ng is running in the container, the docker daemon is collecting the logs from the running container and sending it over the network to the syslog server.

This is why the filtering has to be performed on the network source which is just as you have done.

I setup the following project to demonstrate an example.

.
├── data
│   └── syslog-ng
│       ├── conf
│       │   └── syslog-ng.conf
│       └── logs
│           ├── errors
│           └── syslog
├── docker-compose.yml
└── hack.sh

docker-compose.yml

version: '3.8'

services:
  hack:
    image: alpine
    command: /hack.sh
    logging:
      driver: syslog
      options:
        syslog-address: "tcp://host.docker.internal:601"
    depends_on:
      - syslog
    volumes:
      - '$PWD/hack.sh:/hack.sh'
  syslog:
    container_name: syslog-ng
    image: balabit/syslog-ng:latest
    volumes:
      - '$PWD/data/syslog-ng/conf/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf'
      - '$PWD/data/syslog-ng/logs:/var/log'
    ports:
      - 514:514
      - 601:601

hack.sh: A shell script that logs to stderr and stdout

#!/bin/sh

while :
do
    echo "boom" >/dev/stderr;
    echo "no boom";
    sleep 2;
done

./data/syslog-ng/conf/syslog-ng.conf: filter info to /var/log/syslog and error to /var/log/error

@version: 3.29
@include "scl.conf"

source s_net {
   udp(
     ip("0.0.0.0")
   );

  network(
    ip("0.0.0.0")
    transport(tcp) 
    port(601)
  );

 };
  
  filter f_errors {
    level(err .. emerg);
  };
  filter f_info {
    level(info);
  };

    destination d_file {
      file("/var/log/syslog");
    };
    destination d_error_file {
      file("/var/log/errors");
    };

      log {
        source(s_net);
        filter(f_info);
        destination(d_file); 
      };
      log {
        source(s_net);
        filter(f_errors);
        destination(d_error_file);
      };

Other than creating a different filter for non-error messages, there is little difference between the above syslog-ng configuration and the one you have provided.

I've tested your example with my `docker run` above, but it doesn't seem to work. I've also tried different combinations, but nothing goes into the error file. If you have an example (Dockerfile for syslog-ng and syslog-ng.conf) that works with my `docker run` line, can you please post it? — Alessandro Cappello, Nov 03 '20 at 16:59
@AlessandroCappello Your log filter appears set up the right way. — Oluwafemi Sule, Nov 09 '20 at 05:29
It works! I will check further to see what's wrong in my configuration, but I could quickly see that yours works just great. Thank you very much for your help! — Alessandro Cappello, Nov 09 '20 at 09:00

Severity level on docker logging to syslog

1 Answers1