16

I have a Node.js server application with Express. I would like to log its activity into ElasticSearch and visualize the logs using Kibana.

What would be the right way to do that?

Should I write a log file of json lines and read it with Logstash?

Veve
  • 6,643
  • 5
  • 39
  • 58
Michael
  • 3,206
  • 5
  • 26
  • 44

1 Answers1

18

I'd recommend log4js. It has a range of useful appenders, and logstash is one of them. It works over UDP.

Here is an example taken from the log4js site:

var log4js = require('../lib/log4js');

/*
 Sample logstash config:
   udp {
    codec => json
    port => 10001
    queue_size => 2
    workers => 2
    type => myAppType
  }
*/

log4js.configure({
  "appenders": [
    {
      type: "console",
      category: "myLogger"
    },
    {
      "host": "127.0.0.1",
      "port": 10001,
      "type": "logstashUDP",
      "logType": "myAppType", // Optional, defaults to 'category'
      "fields": {             // Optional, will be added to the 'fields' object in logstash
        "field1": "value1",
        "field2": "value2"
      },
      "layout": {
        "type": "pattern",
        "pattern": "%m"
      },
      "category": "myLogger"
    }
  ]
});

var logger = log4js.getLogger("myLogger");
logger.info("Test log message %s", "arg1", "arg2");
Simon Johnson
  • 7,802
  • 5
  • 37
  • 49
  • 3
    The problem with UDP is that if the ElasticSearch falls for any reason, you start losing data, while with files, you have some buffer on the hard drive before you start losing data. – Michael Jan 07 '16 at 14:33
  • 1
    Another alternative would be to use RabbitMQ or something similar to ensure no log messages are lost. – Skippy Feb 19 '16 at 13:44