2

I have no idea what I did wrong. I'm trying to make right configuration of ActiveMQ Artemis HA cluster on 2 docker containers, but I don't see in logs anything about "backup server" and so on. It seems to not work. I tried to make a simple Java client for the cluster, but it doesn't connect to it. Please don't write "have you read a documentation" - because yes, I've read it. From my perspective it seems like there is no connection between master and slave, but I don't know why. I've configured docker environment with data:

Dockerfile:

FROM ubuntu:latest
LABEL maintainer="Bernard"

RUN mkdir /artemis
WORKDIR /artemis

RUN apt-get update
RUN apt-get -y install wget
RUN apt-get -y install openjdk-11-jdk
RUN java --version

RUN wget -O "artemis.tar" "https://www.apache.org/dyn/closer.cgi?filename=activemq/activemq-artemis/2.18.0/apache-artemis-2.18.0-bin.tar.gz&action=download"
RUN tar -xvf ./artemis.tar; \
ln -s /artemis/apache-artemis-2.18.0/ ./current

RUN /artemis/current/bin/artemis create --user admin --password admin --http-host 0.0.0.0 --require-login --relax-jolokia bernard

EXPOSE 8161 \
9404 \
61616 \
61617 \
5445 \
5672 \
1883 \
61613

ENTRYPOINT ["/artemis/bernard/bin/artemis", "run"]

docker-compose.yml:

version: '3'

services:
  myartemis1:
    build: artemis/
    volumes:
      - "/e/sandbox/artemis_volume/artemis1/etc:/artemis/bernard/etc"
    networks:
      artemis-cluster-network:
        ipv4_address: 172.20.0.100
    ports:
      - 8161:8161
      - 9404:9404
      - 9876:9876
      - 61616:61616
      - 5445:5445
      - 5672:5672
      - 1883:1883
      - 61613:61613
      - 5432:5432
  myartemis2:
    build: artemis/
    volumes:
      - "/e/sandbox/artemis_volume/artemis2/etc:/artemis/bernard/etc"
    networks:
      artemis-cluster-network:
        ipv4_address: 172.20.0.101
    ports:
      - 8162:8161
      - 9405:9404
      - 9877:9876
      - 61618:61616
      - 61617:61617
      - 5446:5445
      - 5673:5672
      - 1884:1883
      - 61614:61613
      - 5433:5432

volumes:
  artemis1:
  artemis2:

networks:
  artemis-cluster-network:
    ipam:
      config:
        - subnet: 172.20.0.0/16

master broker.xml:

<?xml version='1.0'?>
<configuration xmlns="urn:activemq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xi="http://www.w3.org/2001/XInclude" xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
    <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:activemq:core ">
        <bindings-directory>data/bindings</bindings-directory>
        <journal-directory>data/journal</journal-directory>
        <large-messages-directory>data/largemessages</large-messages-directory>
        <paging-directory>data/paging</paging-directory>
        <connectors>
            <connector name="netty-connector">tcp://localhost:61616</connector>
            <connector name="server1-connector">tcp://172.20.0.101:61617</connector>
        </connectors>
        <acceptors>
            <acceptor name="netty-acceptor">tcp://localhost:61616</acceptor>
        </acceptors>
        <ha-policy>
            <replication>
                <master>
                    <group-name>mynode</group-name>
                    <check-for-live-server>true</check-for-live-server>
                    <cluster-name>my-cluster</cluster-name>
                </master>
            </replication>
        </ha-policy>
        <broadcast-groups>
            <broadcast-group name="my-broadcast-group">
                <group-address>231.7.7.7</group-address>
                <group-port>9876</group-port>
                <broadcast-period>2000</broadcast-period>
                <connector-ref>netty-connector</connector-ref>
            </broadcast-group>
        </broadcast-groups>
        <discovery-groups>
            <discovery-group name="my-discovery-group">
                <group-address>231.7.7.7</group-address>
                <group-port>9876</group-port>
                <refresh-timeout>10000</refresh-timeout>
            </discovery-group>
        </discovery-groups>
        <cluster-connections>
            <cluster-connection name="my-cluster">
                <address>jms</address>
                <connector-ref>netty-connector</connector-ref>
                <check-period>1000</check-period>
                <connection-ttl>5000</connection-ttl>
                <min-large-message-size>50000</min-large-message-size>
                <call-timeout>5000</call-timeout>
                <retry-interval>500</retry-interval>
                <retry-interval-multiplier>1.0</retry-interval-multiplier>
                <max-retry-interval>5000</max-retry-interval>
                <initial-connect-attempts>-1</initial-connect-attempts>
                <reconnect-attempts>-1</reconnect-attempts>
                <use-duplicate-detection>true</use-duplicate-detection>
                <message-load-balancing>ON_DEMAND</message-load-balancing>
                <max-hops>1</max-hops>
                <confirmation-window-size>32000</confirmation-window-size>
                <call-failover-timeout>30000</call-failover-timeout>
                <notification-interval>1000</notification-interval>
                <notification-attempts>2</notification-attempts>
                <discovery-group-ref discovery-group-name="my-discovery-group"/>
            </cluster-connection>
        </cluster-connections>
        <security-settings>
            <security-setting match="#">
                <permission type="createNonDurableQueue" roles="guest"/>
                <permission type="deleteNonDurableQueue" roles="guest"/>
                <permission type="createDurableQueue" roles="guest"/>
                <permission type="deleteDurableQueue" roles="guest"/>
                <permission type="createAddress" roles="guest"/>
                <permission type="deleteAddress" roles="guest"/>
                <permission type="consume" roles="guest"/>
                <permission type="browse" roles="guest"/>
                <permission type="send" roles="guest"/>
                <permission type="manage" roles="guest"/>
            </security-setting>
        </security-settings>
        <addresses>
            <address name="DLQ">
                <anycast>
                    <queue name="DLQ"/>
                </anycast>
            </address>
            <address name="ExpiryQueue">
                <anycast>
                    <queue name="ExpiryQueue"/>
                </anycast>
            </address>
            <address name="exampleQueue">
                <anycast>
                    <queue name="exampleQueue"/>
                </anycast>
            </address>
        </addresses>
        <address-settings>
            <address-setting match="#">
                <redistribution-delay>0</redistribution-delay>
            </address-setting>
        </address-settings>
    </core>
</configuration>

slave broker.xml:

<?xml version='1.0'?>
<configuration xmlns="urn:activemq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xi="http://www.w3.org/2001/XInclude" xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
    <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:activemq:core ">
        <paging-directory>data/paging</paging-directory>
        <bindings-directory>data/bindings</bindings-directory>
        <journal-directory>data/journal</journal-directory>
        <large-messages-directory>data/large-messages</large-messages-directory>
        <connectors>
            <connector name="netty-connector">tcp://localhost:61617</connector>
            <connector name="server0-connector">tcp://172.20.0.100:61616</connector>
        </connectors>
        <acceptors>
            <acceptor name="netty-acceptor">tcp://localhost:61617</acceptor>
        </acceptors>
        <ha-policy>
            <replication>
                <slave>
                    <group-name>mynode</group-name>
                    <cluster-name>my-cluster</cluster-name>
                </slave>
            </replication>
        </ha-policy>
        <broadcast-groups>
            <broadcast-group name="my-broadcast-group">
                <group-address>231.7.7.7</group-address>
                <group-port>9876</group-port>
                <broadcast-period>2000</broadcast-period>
                <connector-ref>netty-connector</connector-ref>
            </broadcast-group>
        </broadcast-groups>
        <discovery-groups>
            <discovery-group name="my-discovery-group">
                <group-address>231.7.7.7</group-address>
                <group-port>9876</group-port>
                <refresh-timeout>10000</refresh-timeout>
            </discovery-group>
        </discovery-groups>
        <cluster-connections>
            <cluster-connection name="my-cluster">
                <address/>
                <connector-ref>netty-connector</connector-ref>
                <check-period>1000</check-period>
                <connection-ttl>5000</connection-ttl>
                <min-large-message-size>50000</min-large-message-size>
                <call-timeout>5000</call-timeout>
                <retry-interval>500</retry-interval>
                <retry-interval-multiplier>1.0</retry-interval-multiplier>
                <max-retry-interval>5000</max-retry-interval>
                <initial-connect-attempts>-1</initial-connect-attempts>
                <reconnect-attempts>-1</reconnect-attempts>
                <use-duplicate-detection>true</use-duplicate-detection>
                <message-load-balancing>ON_DEMAND</message-load-balancing>
                <max-hops>1</max-hops>
                <confirmation-window-size>32000</confirmation-window-size>
                <call-failover-timeout>30000</call-failover-timeout>
                <notification-interval>1000</notification-interval>
                <notification-attempts>2</notification-attempts>
                <discovery-group-ref discovery-group-name="my-discovery-group"/>
            </cluster-connection>
        </cluster-connections>
        <security-settings>
            <security-setting match="#">
                <permission type="createNonDurableQueue" roles="guest"/>
                <permission type="deleteNonDurableQueue" roles="guest"/>
                <permission type="createDurableQueue" roles="guest"/>
                <permission type="deleteDurableQueue" roles="guest"/>
                <permission type="createAddress" roles="guest"/>
                <permission type="deleteAddress" roles="guest"/>
                <permission type="consume" roles="guest"/>
                <permission type="browse" roles="guest"/>
                <permission type="send" roles="guest"/>
                <permission type="manage" roles="guest"/>
            </security-setting>
        </security-settings>
        <addresses>
            <address name="DLQ">
                <anycast>
                    <queue name="DLQ"/>
                </anycast>
            </address>
            <address name="ExpiryQueue">
                <anycast>
                    <queue name="ExpiryQueue"/>
                </anycast>
            </address>
            <address name="exampleQueue">
                <anycast>
                    <queue name="exampleQueue"/>
                </anycast>
            </address>
        </addresses>
        <address-settings>
            <address-setting match="#">
                <redistribution-delay>0</redistribution-delay>
            </address-setting>
        </address-settings>
    </core>
</configuration>

snippet of Java client:

final String groupAddress = "231.7.7.7";
final int groupPort = 9876;
DiscoveryGroupConfiguration discoveryGroupConfiguration = new DiscoveryGroupConfiguration();
UDPBroadcastEndpointFactory udpBroadcastEndpointFactory =
        new UDPBroadcastEndpointFactory().setGroupAddress(groupAddress)
        .setGroupPort(groupPort);
discoveryGroupConfiguration.setBroadcastEndpointFactory(udpBroadcastEndpointFactory);
ConnectionFactory jmsConnectionFactory =
        ActiveMQJMSClient.createConnectionFactoryWithHA(discoveryGroupConfiguration, JMSFactoryType.CF);
Connection jmsConnection1 = jmsConnectionFactory.createConnection();
Connection jmsConnection2 = jmsConnectionFactory.createConnection();
Domenico Francesco Bruscino
  • 1,775
  • 7
  • 9
Bernard Burn
  • 661
  • 5
  • 20
  • 2
    The two cluster nodes will try to discover each other via UDP. Each node will listen on and broadcast on `231.7.7.7:9876`. I'm not a Docker expert, but I believe UDP needs to be treated different from TCP. Are you sure you've configured your containers appropriately to allow UDP communication between as well as between the containers and the host itself where your client is running? – Justin Bertram Oct 11 '21 at 19:30

1 Answers1

2

The netty-acceptor acceptor must use the container IP address to allow external connections, i.e. the master acceptor should be:

<acceptor name="netty-acceptor">tcp://172.20.0.100:61616</acceptor>

and the slave acceptor should be:

<acceptor name="netty-acceptor">tcp://172.20.0.101:61616</acceptor>

The netty-connector must use the container IP address so other broker instances are able to connect to it when it is discovered, i.e. the master connector should be:

<connector name="netty-connector">tcp://172.20.0.100:61617</connector>

and the slave connector should be:

<connector name="netty-connector">tcp://172.20.0.100:61617</connector>

Docker Compose makes deploying microservice applications very easy but it has some limitations for a production environment. I would take a look to the open source ArtemisCloud.io project, it is a collection of container images that provide a way to deploy the Apache ActiveMQ Artemis Broker on Kubernetes.

Domenico Francesco Bruscino
  • 1,775
  • 7
  • 9