2

I have a Splunk forwarder managing logs in my production servers, so I really just need to get the output of my node app into a file that Splunk is watching. What is the downside of simply doing the following in production: 

node server.js &> output.log

As oppose to handling the log output inside the node process with some sort of logging module...

m-a-r-c-e-l-i-n-o
  • 2,622
  • 18
  • 26

2 Answers2

2

checkout supervisord which is a logging and babysitting tool which becomes the parent of processes like a node server which can handle redirecting both standard out and standard error to files of your choosing ... besides it will sniff for abends and throw the child process back in when needed

here is a typical config file : /etc/supervisor/conf.d/supervisord.conf

[supervisord]
nodaemon=true
logfile=GKE_MASTER_LOGDIR/supervisord_nodejs_GKE_FLAVOR_USER.log
pidfile=GKE_MASTER_LOGDIR/supervisord_nodejs_GKE_FLAVOR_USER.pid
stdout_logfile_maxbytes = 1MB
stderr_logfile_maxbytes = 1MB
logfile_backups = 50
# loglevel = debug



[program:nodejs]
command=/tmp/boot_nodejs.sh %(ENV_MONGO_SERVICE_HOST)s   %(ENV_MONGO_SERVICE_PORT)s
stdout_logfile = GKE_MASTER_LOGDIR/nodejs_GKE_FLAVOR_USER_stdout.log
stderr_logfile = GKE_MASTER_LOGDIR/nodejs_GKE_FLAVOR_USER_stderr.log
stdout_logfile_maxbytes = 1MB
stderr_logfile_maxbytes = 1MB
logfile_backups = 50
autostart = True
autorestart = True
# user = GKE_NON_ROOT_USER

in my case this all happens inside a Docker container so here is a snippet of my Dockerfile which launches supervisord which in turn launches nodejs and in so doing redirects stdout / err to logging files which supervisord rotates based on space and/or time ... use of Docker is orthogonal to using supervisord so YMMV

CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf" ]

for completeness below I include the boot_nodejs.sh referenced above

#!/bin/bash

given_mongo_service_host=$1
given_mongo_service_port=$2

current_dir=$(dirname "${BASH_SOURCE}")

current_timestamp="timestamp "$(date '+%Y%m%d_%H%M%S_%Z')

echo
echo "______________  fresh nodejs server bounce  ______________  $current_timestamp"
echo

# ............... now output same to standard error so its log gets the hat tip

(>&2 echo )
(>&2 echo "______________  fresh nodejs server bounce  ______________  $current_timestamp" )
(>&2 echo )


# ................

export MONGO_URL=mongodb://$given_mongo_service_host:$given_mongo_service_port

type node

node main.js
Scott Stensland
  • 26,870
  • 12
  • 93
  • 104
  • I appreciate the completeness and up voted your answer, Scott. However, I'm already handling the node.js process recovery portion and have no real concerns on that end. So speaking strictly from a logging standpoint, do you see any downside to simply redirecting the output as I depicted above? – m-a-r-c-e-l-i-n-o Jun 06 '17 at 02:21
  • 1
    What you show is fine too ... assure you do similar to redirect stderr as in ... command >> log_file 2>> err_file – Scott Stensland Jun 06 '17 at 02:25
  • doing only that will ignore capture of stderror ... what you show only logs standard out ... each of these two flavors of output get their own filehandles 1 and 2 ... see details here : https://stackoverflow.com/questions/7526971/how-to-redirect-both-stdout-and-stderr-to-a-file – Scott Stensland Jun 06 '17 at 02:43
  • I understand and I'm fine with it redirecting everything to just one file. I was under the impression that `&>` (emphasis on the `&`) would capture both stdout and stderr. In my local tests here, it appears to be the case. Am I missing something? My knowledge of system admin is a bit limited, please elaborate if you can. – m-a-r-c-e-l-i-n-o Jun 06 '17 at 02:48
  • 1
    cmd >> file.log 2>&1 this will append to end of that file outputs from both stdout and stderr ... enjoy – Scott Stensland Jun 06 '17 at 03:53
1

There's no problem with redirecting your output to a log file. In a lot of ways, this is preferable.

Having your application write logs directly is more useful when your application is complicated and needs a lot of log configuration, possibly writing to several log files. What I do is use Winston for logging. Normally the only log transport enabled is the console, and I can redirect that to a file if I want. But, I also have in my app config a way to specify other transports and config. I use that for writing directly to Logstash and such.

Brad
  • 159,648
  • 54
  • 349
  • 530
  • Thanks for your input, much appreciated. Putting aside the ability to specify custom transports and log levels, what would you say is the practical upside of using Winston over the native console methods (i.e. log, error, warn, etc)? – m-a-r-c-e-l-i-n-o Jun 06 '17 at 02:43
  • @m-a-r-c-e-l-i-n-o The separate transports is the main benefit of Winston. In general, it's a nice performant framework for logging. It also includes formatters, which by default are quite good. (Timestamps and such by default, but you can also format logging any way you want.) – Brad Jun 06 '17 at 02:45
  • Gotcha! Thanks again for the input. Having decent formatting (e.g. timestamps) does sound compelling, but seems to be overkill to use Winston just that. Will probably extend the native console methods to output additional information if Splunk's reconciliation of events and formatting is lacking. – m-a-r-c-e-l-i-n-o Jun 06 '17 at 02:54
  • 1
    @m-a-r-c-e-l-i-n-o Yeah for sure, don't add anything extra that you don't need. Also, I don't know about Splunk's log ingest but I usually log structured data as JSON. This makes indexing and searching much easier later. – Brad Jun 06 '17 at 02:56