EMQX- Publish MQTT Topic with unique identifier is taking much more time than Static MQTT Topic

Question

I was trying to publish messages on emqx broker on different topics.Scenario takes much time while publishing with dynamic topic with one client and if we put topic name as static it takes much less time.

Here I have posted result and code for the same.

I am using EMQX broker with Eclipse paho client Version 3 and Qos level 1.

Time for different topics with 100 simple publish message (Consider id as dynamic here):

Total time pattern 1: /config/{id}/outward::36 sec -----------------> HERE TOPIC is DYNAMIC. and {id} is a variable whose value is changing in loop as shown in below code

Total time pattern 2: /config/test::1.2 sec -----------------------> HERE TOPIC is STATIC

How shall I publish message with different id so topic creation wont take much time?

public class MwttPublish {
    static IMqttClient instance= null;
public static IMqttClient getInstance() {
        try {
            if (instance == null) {         
                instance = new MqttClient(mqttHostUrl, "SimpleTestMQTT");
            }
            if (!instance.isConnected()) {
                MqttConnectOptions options = new MqttConnectOptions();
                options.setUserName("test");
                options.setPassword("test".toCharArray());
                options.setAutomaticReconnect(true);
                options.setCleanSession(false);
                options.setConnectionTimeout(10);

                instance.connect(options);
            }
        } catch (final Exception e) {
            System.out.println("Exception in mqtt: {}" + e.getMessage());
        }
        return instance;
    }
    public static void publishMessage() throws MqttException {
        IMqttClient iMqttClient = getInstance();
        MqttMessage mqttMessage = new MqttMessage("Hello".getBytes());
        mqttMessage.setQos(1);
        mqttMessage.setRetained(true);

        System.out.println("Publish Start for pattern 1");
        int i =0;
        final BigDecimal mqttmsgPublishstartTime = new BigDecimal(System.currentTimeMillis());
        do {
            iMqttClient.publish("/config/" +i +"/outward", mqttMessage);
            i++;
        }while(i<100);
        System.out.println("Total time  pattern 1 /config/i/outward::" + (new BigDecimal(System.currentTimeMillis())).subtract(mqttmsgPublishstartTime));


        System.out.println("Publish Start for pattern 2");

        final BigDecimal mqttmsgPublishstartTime1 = new BigDecimal(System.currentTimeMillis());
        i =0;
        do {
            iMqttClient.publish("/config/test", mqttMessage);
            i++;
        }while(i<100);
        System.out.println("Total time pattern 2 /config/test::" + (new BigDecimal(System.currentTimeMillis())).subtract(mqttmsgPublishstartTime1));

        
    }
}

hardillb · Answer 1 · 2021-12-03T09:48:16.097

0

This is not a valid test, you've fallen into many of the clasic micro benchmark traps e.g.

Way too small a sample size
No account for JVM JIT warm up or GC overhead
Not comparing like to like e.g. time taken to concatenate the strings for the topics

Please check out the following: https://stackoverflow.com/a/2844291/504554

Also from a MQTT point of view topics are ephemeral they only really "exist" for the instant a message is published while the broker checks for subscribed clients with a matching pattern.

edited Dec 03 '21 at 09:48

answered Dec 03 '21 at 09:25

hardillb

54,545
11
67
105

I understand what you are teling about micro benchmark.I will remove those things.But on the logs I can see it works slowly.Now the question is why my publisher is publishing topic slow when topic is dynamic.I understand my qos level is 1 and publisher will wait for puback from broker not from subsriber (correct me if I am wrong here).Still same code works fast when topic is static. – Pranav Shah Dec 03 '21 at 10:08
Can I send 1000 messages on one client only.My job is to send data over mqtt which is in db.If I push all messages over one client with dynamic topic it is taking much time.So I did this micro benchmark.And the problem is topic only as if I do it to static everything works fine. – Pranav Shah Dec 03 '21 at 10:20
You can send as many messages as you want from a single client, how long it takes will depend on many things, but how large the messages are followed by the broker is implemented/configured is likely to be the dominant limiting factors on messages rate – hardillb Dec 03 '21 at 10:47
I have updated my question here can you please look into that once – Pranav Shah Dec 03 '21 at 12:38
It's still not a valid test, you need to pre-compute all the dynamic topics before the publishing loop to make it better. but again, this is not likely to be a client side issue, it's going to be a broker config/design issue. Also try the test with QOS 0 and see what happens. – hardillb Dec 04 '21 at 22:24
That's exactly I want how to precompute topics so topic creation won't take much time Or what should I check at broker side configuration.If any doc or link for both then kindly give so can check my configuration.With QOS 0 it works fast like 100 records in 4 sec. – Pranav Shah Dec 05 '21 at 03:11

EMQX- Publish MQTT Topic with unique identifier is taking much more time than Static MQTT Topic

1 Answers1