Why isn't telegraf reading environmental variables?

Question

My goal is to put my telegraf config into source control. To do so, I have a repo in my user's home directory with the appropriate config file which has already been tested and proven working.

I have added the path to the new config file in the "default" environment variables file:

/etc/default/telegraf

like this:

TELEGRAF_CONFIG_PATH="/home/ubuntu/some_repo/telegraf.conf"

... as well as other required variables such as passwords.

However, when I attempt to run

telegraf --test

It says No config file specified, and could not find one in $TELEGRAF_CONFIG_PATH etc.

Further, if I force it by

telegraf --test --config /home/ubuntu/some_repo/telegraf.conf

Then the process fails because it is missing the other required variables.

Questions:

1. What am I doing wrong?
2. Is there not also a way of specifying a config directory too (I would like to break my file down into separate input files)?
3. Perhaps as an alternative to all of this... is there not a way of specifying additional configuration files to be included from within the default /etc/telegraf/telegraf.conf file? (I've been unable to find any mention of this in documentation).

Answer 1

1. What am I doing wrong?

See what user:group owns /etc/default/telegraf. This file is better used when running telegraf as a service via systemd. Additionally, if you run env do you see the TELEGRAF_CONFIG_PATH variable? What about your other variables? If not, then you probably need to source the file first.

2. Is there not also a way of specifying a config directory too (I would like to break my file down into separate input files)?

Yes! Take a look at all the options of telegraf with telegraf --help and you will find:

  --config-directory <directory> directory containing additional *.conf files

3. Perhaps as an alternative to all of this... is there not a way of specifying additional configuration files to be included from within the default /etc/telegraf/telegraf.conf file? (I've been unable to find any mention of this in documentation).

That is not the method I would suggest going down. Check out the config directory option above I mentioned.

Answer 2

Ok, after a LOT of trial and error, I figured everything out. For those facing similar issues, here is your shortcut to the answer:

Firstly, remember that when adding variables to the /etc/default/telegraf file, it must effectively be reloaded. So for example using ubuntu systemctl, that requires a restart.

You can verify that the variables have been loaded successfully using this:

$ sudo strings /proc/<pid>/environ

where <pid> is the "Main PID" from the telegraf status output

Secondly, when testing (eg telegraf --test) then (this is the part that is not necessarily intuitive and isn't documented) you will have to ALSO load the same environmental variables into the current user (eg: SET var=value) such that running

$ env

shows the same results as the previous command.

Hint: This is a good method for loading the current env file directly rather than doing it manually.

Answer 3

Add env vars to /etc/default/telegraf file in order to access them in telegraf.conf file.

https://github.com/influxdata/telegraf/blob/master/docs/CONFIGURATION.md