JSON script to input data from jmxtrans to InfluxDB

Question

I'm trying to use jmxtrans to collect JVM metrics and would need to input the data into InfluxDB. Though I was able to collect data, I'm not sure of the tags and syntax required to input the data into the DB.

I've tried the JSON tutorials as well as the method to write into influx DB:https://docs.influxdata.com/influxdb/v0.8/api/reading_and_writing_data/ but unable to find specific answers, hence the post.

Need help with modifying json files to input data into InfluxDB. Please advise. Example code to retrieve heapmemory usage of another server:

{
 "servers" : [ {
 "port" : "<PORT>",
 "host" : "<IP>",
 "queries" : [ {
  "obj" : "java.lang:type=Memory",
  "attr" : [ "HeapMemoryUsage", "NonHeapMemoryUsage" ],
  "outputWriters" : [ {
    "@class" : "com.googlecode.jmxtrans.model.output.GraphiteWriter",
    "settings" : {
        "templateFile" : "heapmemory-rrd-template.xml",
        "outputFile" : "target/heap.rrd",
        "binaryPath" : "/opt/local/bin",
        "debug" : true,
        "generate" : true
      } 
     } ]
   } ]
 } ]
}

score 0 · Answer 1 · answered Feb 08 '16 at 15:09

0

There is an example of exactly this problem on the jmxtrans Wiki - note this example uses the specific InfluxDbWriter not the GraphiteWriter

answered Feb 08 '16 at 15:09

Simon Hutchinson

47
1
5

Thank you Simon! Much appreciate your help. Any idea how we could make use of the newly added output writers within jmxtrans? I couldn't find the linking document that would let me start using the new java files. – VishalR Feb 09 '16 at 09:35

score 0 · Answer 2 · answered Feb 08 '16 at 15:26

0

I notice that you are using v0.8 which is deprecated right now. v0.9 is not backward compatible with v0.8. So I suggest you to pick the latest version if possible since the structure to write the data differs between these two versions.

In case you have to use v0.8 then here you go :

{
        "name": "cpu_util",
        "columns": [
          "time",
          "sequence_number",
          "avg",
          "unit"
        ],
        "points": [
          [
            1421024460734,
            124666640001,
            74.31932,
            "%"
          ],
          [
            1421024460734,
            124666550001,
            0.7899716,
            "%"
          ]
        ]
      }

"name" --> time series' name

"columns" --> columns in the time series "name"

"points" --> data points cprresponding to the columns mentioned above

Source: https://github.com/icclab/cyclops-udr/wiki/OpenStack

answered Feb 08 '16 at 15:26

Srikanta

1,145
2
12
22

Thank you Srikanta, unfortunately we have to continue using v8.0 for now. Looking for an inbuilt output writer(that I found recently: InfluxDbWriterFactory) that would send values to InfluxDb on the fly. – VishalR Feb 09 '16 at 09:28
under the hood, it makes an API call over the HTTP. So if you have problems with the writerFactory then you could as well make a REST call to injest/retrieve the data – Srikanta Feb 09 '16 at 09:34
Let me explain the scenario better: Right now I have a script that parses the logs and inserts values into InfluxDB using its REST api call. But considering the size of the logs/future enhancements required off the metrics, it would become quite an overhead to use a parsing script. Hence InfluxDBWriterFactory, which I believe would send the data of the jmxtrans straight to the database. Appreciate your suggestions. – VishalR Feb 09 '16 at 10:48

score 0 · Answer 3 · answered Oct 30 '16 at 21:21

I'm working on a similar solution, whereby I am using jmxtrans to poll the JVMs running on each host. JMXTrans sends the results in "enhanced" statsD form to a Telegraf agent on that host. That Telegraf agent sends the metrics on to InfluxDB. (In the next Telegraf version they will support collection from many telegraf agents to one centralized instance). This keeps the demand on the InfluxDB ingestion as low as possible.

I wrote a new JmxTrans writer that is included in the latest snapshot builds. An example of what it sends to telegraf is: JVMMemory,jmxport=1234,attribute=NonHeapMemoryUsage,resultKey=committed:12345|c

This takes advantage of the tagging features of Telegraf/InfluxDB, and supports the sampling features and simplicity of StatsD

JSON script to input data from jmxtrans to InfluxDB

3 Answers3