I’m on influxdb2 (cloud) and i’m getting a really weird situation:
I’m using telegraf’s tail plugin to collect logs and push them to a bucket, my timestamp (which is written in epoch) gets reformatted to ns and telegraf is adding random offset to it, here’s an example:
My input log file contains:
2021-04-08 22:49:07.638339 1617922080 binance ETH_USDT 1m 2067.96 2070.0 2067.96 2069.03 356.11558
2021-04-08 22:49:07.638347 1617922080 binance BNB_USDT 1m 416.8289 417.0972 416.8114 416.95 355.811
2021-04-08 22:49:07.638355 1617922080 binance BTC_USDT 1m 57818.78 57836.06 57818.77 57823.99 24.413998
And telegraf echoes the following lineprotocol lines:
binance,market=ETH_USDT base_volume=356.11558,lowest=2067.96,close=2069.03,open=2067.96,highest=2070 1617922080005000000
binance,market=BNB_USDT lowest=416.8114,close=416.95,highest=417.0972,open=416.8289,base_volume=355.811 1617922080006000000
binance,market=BTC_USDT close=57823.99,base_volume=24.413998,open=57818.78,highest=57836.06,lowest=57818.77 1617922080007000000
From the above sample, my input timestamp is 1617922080 must be transformed by telegraf to ns as 1617922080000000000 but i keep getting some random/unknown offset: 1617922080007000000.
This is my telegraf config:
[[outputs.file]]
files = ["stdout"]
data_format = "influx"
precision = "s"
[[inputs.tail]]
files = ["/data/ohlcv.log"]
tagexclude = ["path", "host"]
from_beginning = true
data_format = "grok"
grok_patterns = ["%{TIMESTAMP_ISO8601:created_at:drop} %{NUMBER:timestamp:ts-epoch} %{WORD:exchange:measurement} %{WORD:market:tag} %{WORD:interval:drop} %{NUMBER:open:float} %{NUMBER:highest:float} %{NUMBER:lowest:float} %{NUMBER:close:float} %{NUMBER:base_volume:float}"]
grok_timezone = "UTC"
Where did this offset come from ?
