How would a human read a json winston log file?

Question

It seems nice for API's, scripts and what not. But reading a winston json stack trace is very hard with a text editor. E.g.

{"level":"info","message":"starting","timestamp":"2014-05-14T15:45:44.334Z"}
{"date":"Wed May 14 2014 08:45:45 GMT-0700 (Pacific Daylight Time)","process":{"pid":8804,"uid":null,"gid":null,"cwd":"C:\\data\\mytool","execPath":"C:\\Program Files\\nodejs\\node.exe","version":"v0.10.21","argv":["node","C:\\data\\mytool\\server"],"memoryUsage":{"rss":45199360,"heapTotal":32171264,"heapUsed":15158096}},"os":{"loadavg":[0,0,0],"uptime":70496.6138252},"trace":[{"column":null,"file":null,"function":"Object.parse","line":null,"method":"parse","native":true},{"column":32,"file":"C:\\data\\mytool\\src\\status.js","function":"Request._callback","line":166,"method":"_callback","native":false},{"column":22,"file":"C:\\data\\mytool\\node_modules\\request\\request.js","function":"Request.self.callback","line":122,"method":"self.callback","native":false},{"column":17,"file":"events.js","function":"Request.EventEmitter.emit","line":98,"method":"EventEmitter.emit","native":false},{"column":14,"file":"C:\\data\\mytool\\node_modules\\request\\request.js","function":"","line":888,"method":null,"native":false},{"column":20,"file":"events.js","function":"Request.EventEmitter.emit","line":117,"method":"EventEmitter.emit","native":false},{"column":12,"file":"C:\\data\\mytool\\node_modules\\request\\request.js","function":"","line":839,"method":null,"native":false},{"column":20,"file":"events.js","function":"IncomingMessage.EventEmitter.emit","line":117,"method":"EventEmitter.emit","native":false},{"column":16,"file":"_stream_readable.js","function":null,"line":920,"method":null,"native":false},{"column":13,"file":"node.js","function":"process._tickCallback","line":415,"method":"_tickCallback","native":false}],"stack":["SyntaxError: Unexpected end of input","    at Object.parse (native)","    at Request._callback (C:\\data\\mytool\\src\\status.js:166:32)","    at Request.self.callback (C:\\data\\mytool\\node_modules\\request\\request.js:122:22)","    at Request.EventEmitter.emit (events.js:98:17)","    at Request.<anonymous> (C:\\data\\mytool\\node_modules\\request\\request.js:888:14)","    at Request.EventEmitter.emit (events.js:117:20)","    at IncomingMessage.<anonymous> (C:\\data\\mytool\\node_modules\\request\\request.js:839:12)","    at IncomingMessage.EventEmitter.emit (events.js:117:20)","    at _stream_readable.js:920:16","    at process._tickCallback (node.js:415:13)"],"level":"error","message":"uncaughtException: Unexpected end of input","timestamp":"2014-05-14T15:45:45.228Z"}

MichaelS · Answer 1 · 2014-12-28T14:24:20.453

24

Simply set the file transport "json" property to false, and you'll get a human readable log. Same as you see in the console.

    var winston = require('winston');
    var logger = new winston.Logger({
      transports: [
        new winston.transports.File({
          json: false,
          filename:'log.log'
        }),
        new winston.transports.Console()
      ],
      exitOnError: false
    });
   logger.log('info', 'some msg');

edited Dec 28 '14 at 14:24

answered Dec 28 '14 at 12:24

MichaelS

7,023
10
51
75

13

Just to clarify - I haven't accepted this answer because it removes the json from the log. I want to keep the json log but use a viewer to make reading it easier. – ubershmekel Jul 19 '15 at 14:16
@ubershmekel Have you found the solution now? I also need to see pretty printed logs in Winston but stored as json format. It seems bunyan cli has that ability but I don't know if Winston has similar feature. – Keming Feb 19 '18 at 20:41
I use `bunyan` to log which comes with a command line tool you can use which behaves like a colorized `less` for its json logs. Maybe the cli works with winston logs too, not sure. – ubershmekel Feb 21 '18 at 22:52

score 17 · Answer 2 · answered May 12 '15 at 16:45

17

Pass it through jq, which is like sed for JSON. E.g.:

jq . file.log

answered May 12 '15 at 16:45

weiyin

6,819
4
47
58

3

To tail the log file, use `tail -f file.log | jq --unbuffered .`. – Resigned June 2023 Jun 16 '17 at 22:29

score 4 · Answer 3 · edited May 23 '17 at 12:34

4

Why not just run it through a JSON formatter on the command line ?

e.g. (example from the link above)

echo '{ element0: "lorem", element1: "ipsum" }' | python -mjson.tool

An alternative may be to look at building a shell script around the above tool (or perhaps) jq to perform some custom stack trace parsing

edited May 23 '17 at 12:34

Community

1
1

answered May 14 '14 at 16:10

Brian Agnew

268,207
37
334
440

Thanks for that bit, though `python -m json.tool` only knows how to digest one line. Still, looking at the results e.g. http://pastebin.com/16AE7Qn7 isn't exactly as nice as looking at a text log stack trace. Not to mention having to scroll through a million of these. – ubershmekel May 14 '14 at 16:15
It would perhaps be worth looking at building a shell script around the above tool (or perhaps) *jq* to perform some custom stack trace parsing. Answer amended... – Brian Agnew May 14 '14 at 16:17
As of Python 3.8, `json.tool` now accepts the json-lines format via the option `--json-lines` so that each new line gets interpreted as a separate json object: https://docs.python.org/3/library/json.html#cmdoption-json-tool-json-lines – alkasm Jan 28 '20 at 01:10

Vikas Kapadiya · Answer 4 · 2017-01-31T18:42:31.480

4

You should try winston-logs-display .

Demo Output:

Also Log.io is good option for this. it supports winston log.

edited Jan 31 '17 at 18:42

answered Jan 31 '17 at 18:35

Vikas Kapadiya

156
9

score 2 · Answer 5 · answered Jul 03 '15 at 10:53

2

If you use Keen.IO - their CLI tool can upload the line-deliminated JSON, then you can use their 'Explorer' to filter/view log events.

keen events:add --collection myLogs --file winston-output.json

answered Jul 03 '15 at 10:53

simbolo

7,279
6
56
96

score 1 · Answer 6 · answered Mar 08 '16 at 05:28

1

Seems node's bunyan has features that let you filter and view json logs in a human readable way with a CLI.

$ node hi.js | bunyan -l warn
[2013-01-04T19:08:37.182Z]  WARN: myapp/40353 on banana.local: au revoir (lang=fr)

answered Mar 08 '16 at 05:28

ubershmekel

11,864
10
72
89

You're right and I haven't accepted my answer because of that. Though it does solve my need to log json and consume that json. – ubershmekel Apr 06 '16 at 01:16

score 1 · Answer 7 · answered Jan 28 '17 at 23:34

It's slow but your shell can do it, get formatted, colourized JSON.

./thing | ndjson

How?

You run some JSON formatting command on each line, bash or zsh syntax is:

./thing | while read in ; do echo "$in" | python -m json.tool ; done

For fish the syntax is

./thing | while read in; echo "$in" | python -mjson.tool; end #fish

To make it extra fancy, just pip install pygments.

Define a handy alias pp, so you run cat file.json | pp.

alias pp="python -mjson.tool | pygmentize -l js"

And then define ndjson

alias ndjson='while read in; do echo "$in" | pp; done'

Now you can type the following to get formatted, colourized JSON.

./thing | ndjson

(use funced and funcsave to define alias in fish)

score 1 · Answer 8 · answered Jan 28 '20 at 00:32

In case it helps (so many years after the last response), I synthesized the one-line command which allows even python 2.7 to take the live-one-line-at-a-time output of tail -f winston.log and pipe it through python's json.tool for pretty-printing.

tail -f winston.log | while read -r line; do echo -n "$line" | python -m json.tool; done

(Note that if you are using python 3.8 and beyond, the parameter --json-lines, when supplied to json.tool, should eliminate the need of the while..do potion).

score 1 · Answer 9 · answered Feb 19 '21 at 17:33

I know I am late on this but there is an easy way.

Open a new tab in your browser
Open devtools
Run this code

const entries = prompt('Log Data').split('\n').map(JSON.parse);

console.table(entries);

Enter your log data into the prompt and press enter.

Now you have an interactive table view of the log entries. You can click the headers to sort and even right click for some options. You can also resize columns.

To view certain entries, do

console.log(entries.filter(entry => entry.foo.bar > 323)); // Just an example, you can do anything with the array

View devtools in full screen to see more of the data.

score 1 · Answer 10 · answered Mar 30 '21 at 16:20

I use lnav: I supports json-logs to some extend: i.e. you can easily define your own log-format. This way you can optimize the display to your needs (i.e. so that it matches your winston log-output.

Here is a simple example for our lnav-log-config:

{
  "$schema": "https://lnav.org/schemas/format-v1.schema.json",
  "winston": {
    "title": "My Custom Json log format",
    "description": "Custom JSON-log format for winston",
    "url": "https://github.com/winstonjs/winston#formats",
    "file-pattern": "\\.jsonlog",
    "json": true,
    "level-field": "level",
    "timestamp-field": "timestamp",
    "body-field": "message",
    "line-format": [
      {
        "field": "__timestamp__"
      },
      " ",
      {
        "field": "__level__",
        "text-transform": "uppercase"
      },
      " ",
      {
        "field": "message"
      }
    ]
  }
}

NOTE: that I use "file-pattern": "\\.jsonlog",: change or remove this as needed.

lnav even works in my WSL2.
Installation as usual: sudo apt install lnav

score 0 · Answer 11 · answered Jul 21 '21 at 09:44

0

I resolved this issue by writing a npm package (usage: tail -f logFile.log | npx winston-log-viewer) you can see full description here

https://stackoverflow.com/a/68455246/4650625

answered Jul 21 '21 at 09:44

Mohammad Ranjbar Z

1,487
1
10
20

score 0 · Answer 12 · answered Nov 18 '21 at 11:26

If you want to visualize winston json log at console you can use munia-pretty-json

Your json data (app-log.json)

{"time":"2021-06-09T02:50:22Z","level":"info","message":"Log for pretty JSON","module":"init","hostip":"192.168.0.138","pid":123}
{"time":"2021-06-09T03:27:43Z","level":"warn","message":"Here is warning message","module":"send-message","hostip":"192.168.0.138","pid":123}

Run the command:

munia-pretty-json app-log.json

Here is readable output on console:

You can format the output with the template. The default template is '{time} {level -c} {message}'

Using template:

munia-pretty-json -t '{module -c} - {level} - {message}' app-log.json

Output:

How would a human read a json winston log file?

12 Answers12

How?

Linked