4

I have a question on which I am stuck and I am not quite sure how to resolve it.

In my work project I have an ActiveMQ queue and I want to send some metrics to Prometheus which will help me to create some alerts in Grafana. I know that for ActiveMQ Artemis I can use this plugin, but I don't understand 100% how to configure it.

My application is deployed on a Kubernetes cluster and the ActiveMQ broker is there too. So I have created ActiveMQPrometheusMetricsPlugin class which implements org.apache.activemq.artemis.core.server.metrics.ActiveMQMetricsPlugin. Now is where I get confused right now I should deploy my application and the metrics would be gather by Prometheus? I should do more configuration?

We usually do not build the application on local env. We are using a pipeline which is building and deploying the app to various envs (dev, test, prod). I should do the configuration similar with the GitHub plugin project, deploy it, and after that find those jars on Kubernetes and move them to the correct location? Also dev-ops said to me that we are using a default conf. I don't know if we have a broker.xml file.

moonkotte
  • 3,661
  • 2
  • 10
  • 25
AdrianMirica
  • 45
  • 1
  • 6
  • What exactly don't you understand about configuring the Artemis Prometheus Metrics Plugin? Why are you creating your own plugin when one already exists (and works)? – Justin Bertram Jun 02 '21 at 14:18
  • Also, what version of ActiveMQ are you using? – Justin Bertram Jun 02 '21 at 14:19
  • What does your dev-ops mean by "default conf"? If you don't have a `broker.xml` how is the broker being configured? – Justin Bertram Jun 02 '21 at 14:21
  • How specifically does your application relate to Prometheus gathering metrics from the broker? Generally speaking those two things are 100% independent. – Justin Bertram Jun 02 '21 at 14:53
  • @JustinBertram the ActiveMQ version it is activemq-artemis:2.16-alpine-latest. The dev-ops said to me that he run the activemq without any broker.xml file to see if it is working. For the first question, I'm not quite sure I understand, I should clone the plugin repo, build it and deploy where it is saying in the readme file? We want to gather these metrics in prometheus to use them in grafana to generate some graphs and some alerts based on them. – AdrianMirica Jun 03 '21 at 08:40
  • How does your dev-ops "run the activemq without any broker.xml"? That does make sense to me. To my knowledge the broker can't run without a `broker.xml` file. – Justin Bertram Jun 03 '21 at 14:10
  • In my first comment I had 2 questions. I asked for clarification about what exactly you don't you understand about configuring the Artemis Prometheus Metrics Plugin. I don't see an answer to this question. I also asked why you are creating your own plugin because you say that you "have created ActiveMQPrometheusMetricsPlugin class." I can only assume that's a class that you have written because there is no class in the Artemis Prometheus Metrics Plugin with that name. I assumed you created your own because you couldn't determine how to configure the Artemis Prometheus Metrics Plugin. – Justin Bertram Jun 03 '21 at 14:16
  • Yes, I don't know how to configure it and I though that I should make a similar class in my project. So how I should integrate the plugin in my project ? – AdrianMirica Jun 03 '21 at 14:38
  • I should clone the repo, build it locally and after that to do the configuration from the readme file? – AdrianMirica Jun 03 '21 at 14:50
  • At this point I recommend you clarify your actual question. Currently it's not at all clear what you're asking. Are you simply wanting to build, deploy, and configure the Artemis Prometheus Metrics Plugin? If so, please make that clear and remove all the other superfluous and irrelevant information. – Justin Bertram Jun 03 '21 at 15:35
  • Yes, because from the readme file I did not understand how to do it. I thought that extra information would be helpful. – AdrianMirica Jun 03 '21 at 15:43
  • This goes back to my first comment. Can you clarify what *exactly* you didn't understand? Where are you getting stuck (e.g. building, deploying, configuring)? Please clarify. – Justin Bertram Jun 03 '21 at 15:49

4 Answers4

6

There are a couple of important points to understand before getting started:

  • When using the Artemis Prometheus Metrics Plugin neither the broker nor the applications "send" metrics to Prometheus. Prometheus itself must retrieve or "scrape" metrics from the broker. This is why the plugin comes with a servlet. The servlet exposes an HTTP endpoint that Prometheus can use to scrape metrics.
  • The Artemis Prometheus Metrics Plugin is part of the broker infrastructure. It is not to be deployed as part of an application. The plugin's jar and war files are deployed on the broker and configured in broker.xml and bootstrap.xml respectively.

The Artemis Prometheus Metrics Plugin provides integration with Prometheus using two modules:

  • artemis-prometheus-metrics-plugin: This provides the actual implementation of org.apache.activemq.artemis.core.server.metrics.ActiveMQMetricsPlugin and packages it with the Micrometer and Prometheus dependencies in an "uber" jar.

  • artemis-prometheus-metrics-plugin-servlet: This provides a war file containing a simple servlet which can be deployed to the broker's embedded web server which then Prometheus can use to scrape metrics.

Once you clone the Artemis Prometheus Metrics Plugin repository simply run mvn install to build these two modules. The output will be in their respective target directories.

After building the modules follow these steps to deploy and configure the Artemis Prometheus Metrics Plugin. If you have some kind of dev-ops group which manages and configures your broker then they would follow these steps.

  1. Copy artemis-prometheus-metrics-plugin/target/artemis-prometheus-metrics-plugin-<VERSION>.jar to <ARTEMIS_INSTANCE>/lib.

  2. Add this to your <ARTEMIS_INSTANCE>/etc/broker.xml:

    <metrics-plugin class-name="org.apache.activemq.artemis.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/>

  3. Create the directory <ARTEMIS_INSTANCE>/web.

  4. Copy artemis-prometheus-metrics-plugin-servlet/target/metrics.war to <ARTEMIS_INSTANCE>/web.

  5. Add this to the web element in <ARTEMIS_INSTANCE>/etc/bootstrap.xml:

    <app url="metrics" war="metrics.war"/>
Justin Bertram
  • 29,372
  • 4
  • 21
  • 43
  • Thank you so much for all your time explaining me this things. I do not have any prior experience to this kind of stuff. I will try tomorrow and I hope that everything will go alright. – AdrianMirica Jun 03 '21 at 16:35
0

artemis-prometheus-metrics-plugin is a good alternative however not work good in clustered environment with Colocated replication:

2021-11-16 12:15:46,374 WARN  [org.apache.activemq.artemis.core.server] AMQ222261: Failed to activate a backup: java.security.PrivilegedActionException: java.io.NotSerializableException: io.micrometer.prometheus.PrometheusMeterRegistry
    at java.base/java.security.AccessController.doPrivileged(Native Method) [java.base:]
    at org.apache.activemq.artemis.core.config.impl.ConfigurationImpl.copy(ConfigurationImpl.java:2328) [artemis-server-2.19.0.jar:2.19.0]
    at org.apache.activemq.artemis.core.server.cluster.ha.ColocatedHAManager.activateReplicatedBackup(ColocatedHAManager.java:186) [artemis-server-2.19.0.jar:2.19.0]
    at org.apache.activemq.artemis.core.server.cluster.ha.ColocatedHAManager.activateBackup(ColocatedHAManager.java:99) [artemis-server-2.19.0.jar:2.19.0]
    at org.apache.activemq.artemis.core.server.impl.ColocatedActivation$1.handlePacket(ColocatedActivation.java:127) [artemis-server-2.19.0.jar:2.19.0]
    at org.apache.activemq.artemis.core.server.cluster.ClusterController$ClusterControllerChannelHandler.handlePacket(ClusterController.java:469) [artemis-server-2.19.0.jar:2.19.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:820) [artemis-core-client-2.19.0.jar:2.19.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:428) [artemis-core-client-2.19.0.jar:2.19.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:396) [artemis-core-client-2.19.0.jar:2.19.0]
    at org.apache.activemq.artemis.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:688)
Jairo Gutierrez
  • 1
0

In addition to the accepted answer I would add that the metrics-plugin tag became in the meantime obsolete. A valid configuration in broker.xml would be:

<metrics>
    <jvm-gc>true</jvm-gc>
    <jvm-memory>true</jvm-memory>  
    <jvm-threads>true</jvm-threads>
    <plugin class-name="com.redhat.amq.broker.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/>
</metrics>

The specified class name matches the plugin from https://github.com/rh-messaging/artemis-prometheus-metrics-plugin which you can directly download if you select a release tag, like v2.0.0, you don't have to compile it by yourself.

Please note that the 'jvm' tags are optional, you can remove them.

Cristian Chereji
  • 66
  • 3
0

I stumbled upon this thread since I was trying to do this exact setup for Artemis MQ 2.19.1. I am adding a few points below which helped me complete.

  • In Artemis MQ {instance}/etc/broker.xml file, i changed it to the following
<metrics>
    <plugin class-name="org.apache.activemq.artemis.core.server.metrics.plugins.ArtemisPrometheusMetricsPlugin"/>
</metrics>
  • Assuming the Prometheus and Grafana setup is already done, add the following to the prometheus.yml file
- job_name: "artemismq"
  scrape_interval: 5s
  static_configs:
    - targets: ["localhost:8161"]

This should get you up and running. You can then modify it as required

Huge thanks to Justin for his contributions.

Mayukh Chakravartti
  • 5
  • 1