How to clear ALL retained mqtt messages from Mosquitto?

Question

I've seen the mosquitto_pub -h [server] -r -n -t [XYZ] syntax for clearing out one off messages. My problem is the device developers have posted a lot of garbage messages.

I have a Java/Paho code base that I'd like to modify to do this automatically as needed, but I can't seem to publish a zero byte message. I tried

client.publish(topic,null);

...but that didn't seem to work.

Any suggestions on how to delete everything, en mass?

This might help: `mosquitto_sub -t '#' --remove-retained --retained-only`. See also: https://mosquitto.org/man/mosquitto_sub-1.html — rel, May 01 '20 at 17:36

Gussoh · Answer 1 · 2023-06-22T07:55:23.747

25

Here is how to do it properly with a shell script.

#!/bin/sh
echo "cleaning " $1 " :: usage: cleanmqtt <host>"
mosquitto_sub -h $1 -t "#" -v --retained-only | while read line; do mosquitto_pub -h $1 -t "${line% *}" -r -n; done

Just put it in a file called somthing like

cleanmqtt.sh

Then run

sh cleanmqtt.sh localhost

This solution is quite crude. You cant specify what to delete or anything. You may have to abort with ctrl-c after you can assume that it has received all the messages.

edited Jun 22 '23 at 07:55

answered Jan 29 '17 at 21:54

Gussoh

799
1
6
13

Could you show how to modify the script to include username and password? When I run it, I get "Connection Refused: not authorised". Thanks. – moster67 Nov 21 '17 at 08:38
1

Sure, add -u _username_ -P _password_ in the line to both the mosquitto_sub and mosquitto_pub – Gussoh Nov 22 '17 at 08:51
2

If your topic has spaces in it, you can change the command to: `mosquitto_sub -h $1 -t "#" -v | while read line; do mosquitto_pub -h $1 -t "${line% *}" -r -n; done` The change is to remove the `_` after `read line` and change `$line` to `${line% *}` – Robert J Berger Dec 30 '18 at 08:52
Cool Robert, I changed to your suggestion – Gussoh Jan 05 '19 at 14:32

hardillb · Accepted Answer · 2016-04-19T21:19:56.590

9

There are 2 options for this using the paho client code depending on which of the 2 publish methods you use.

MqttMessage msg = new MqttMessage(new byte[0]);
msg.setRetained(true);
client.publish(topic, msg);

or

client.publish(topic, new byte[0],0,true);

The other option would be to stop mosquitto and delete the persistence file and restart

edited Apr 19 '16 at 21:19

answered Apr 19 '16 at 21:12

hardillb

54,545
11
67
105

1

Actually, first version will not work, as the message needs to be published with the retained flag set. – kartben Apr 19 '16 at 21:18
2

Deleting the persistence file turned out to be the option I really needed. – JohnL Apr 20 '16 at 14:03
1

This is how I deleted the persistence file in Ubuntu 16.04: `sudo service mosquitto stop` `sudo rm /var/lib/mosquitto/mosquitto.db` `sudo service mosquitto start` You can check in your **mosquitto.conf** where your **persistence_file** is located. – Tobias Holm Jul 04 '17 at 15:02
2

It is important to understand that deleting the persistence file also wipes out all queued messages and any persistent subscriptions. It is better to clear individual topics with publishing a null message in a production environment – hardillb Jul 04 '17 at 15:05
@TobiasHolm Sovle my problem! Thanks!! – JackWu Dec 01 '17 at 11:20

score 8 · Answer 3 · edited Apr 14 '23 at 08:38

8

Mosquitto client provides --remove-retained option:

mosquitto_sub -h $host --remove-retained -t '#' -W 1

Tunning -t can handle specific topics to be cleared.

Edit: (as suggested by Nik in the first comment) Don't use it in connection with -E as Mosquitto would exit immediately without sending the removals. Using -W 1 waits one second and then the client terminates.

edited Apr 14 '23 at 08:38

Patrick Roocks

3,129
3
14
28

answered Dec 17 '21 at 10:37

jonasmike

183
2
8

2

This is the most efficient way, however, when using `-E` the client exits immediately after `Client X received SUBACK` i.e. after subscribing, but before receiving any messages, so actual removal doesn't happen. The only reliable way to make the client disconnect and exit that I found is using `-W 1` flag, i.e. process messages for 1 sec then exit. – Nik Jun 13 '22 at 16:26

score 2 · Answer 4 · answered Jul 01 '19 at 15:38

2

Since I don't have enough points to comment, running

#!/bin/sh
echo "cleaning " $1 " :: usage: cleanmqtt <host>"
mosquitto_sub -h $1 -t "#" -v | while read line; do mosquitto_pub -h $1 -t "${line% *}" -r -n; done

could cause an infinite loop due to pub/sub delays. Adding --retained-only to mosquitto_sub seems to remove the infinite loop.

answered Jul 01 '19 at 15:38

user3084651

21
3

Nice! I added it. This also solves the ctrl-c issue. There is also a --remove-retained but I can't seem to understand how that works. – Gussoh Sep 09 '20 at 09:53

score 1 · Answer 5 · edited Jan 09 '19 at 16:40

1

This should work:

client.publish(topic, new byte[]{}, 0, true);

Also, you may be interested in this script from Eclipse Paho Python, to clear a given topic hierarchy. You may want to implement a similar behavior in Java, but it looks like you may be looking for a one-off solution, so maybe just use the Python script :)

edited Jan 09 '19 at 16:40

rve

5,897
3
40
64

answered Apr 19 '16 at 21:15

kartben

2,485
21
24

The python script doesnt seem to work with non-ascii topics or payloads. The shell script I just posted seems to work better. – Gussoh Jan 29 '17 at 21:56
"this script" link is now 404 – Trevor Jul 11 '17 at 08:58

score 1 · Answer 6 · answered Jun 12 '19 at 14:56

If you are using the Mosquitto MQTT Broker, disable "Retained messages" using the official method, provided by Mosquitto

First find mosquitto.conf file

(In my Ubuntu/EC2 instance it is stored in /etc/mosquitto directory, I assume your mosquitto.conf file path is /etc/mosquitto/mosquitto.conf)

Edit with your favorite text editor, mine is nano.

sudo nano /etc/mosquitto/mosquitto.conf

and in that file replace "persistence false" in place of "persistence true"

persistence false

Now save file (if using nano press ctrl+o and then enter to save, ctrl+x to exit)

now restart mosquitto using below commands

sudo service mosquitto stop
sudo service mosquitto start

Note: If this config path does not exist in your case, find config file using this command-

sudo find / -name mosquitto.conf

That just stops retained messages being stored, across restarts, it doesn't actually clear them (in a running broker) — hardillb, Jun 12 '19 at 16:03
Agreed, for that remove/re-create db file (manually or by such code:file_operation) which is stored in persistence_location (or in my case /var/lib/mosquitto/mosquitto.db) — Manav Akela, Jun 12 '19 at 17:30

score 1 · Answer 7 · answered Jan 01 '20 at 22:58

for powershell users

had this issue on windows, so here with powershell. Mosquitto needs to be installed on the command host.

get retained messages

I did not want to clear all retained messages. Example, only the ones containing "octo" in it's topic. Let's see what's there:

mosquitto_sub.exe -h <mqtt host> -v -u <mqtt user> -P <mqtt password> -t '#' --retained-only|
Select-String octo

(replace mqtt host, user, password as needed)

delete retained messages

use the same search string here ( in this example "octo"):

mosquitto_sub.exe -h <mqtt host> -v -u <mqtt user> -P <mqtt password> -t '#' --retained-only|
Select-String octo |
Out-String -Stream|ForEach-Object -Process {$_.Split(" ")[0]}|
%{mosquitto_pub.exe -h <mqtt host> -u <mqtt user> -P <mqtt password> -t "$_" -r -n}

Again, replace all occurences of mqtt host, user, password.

Simply check again with step 1 if anything is left :)

score 0 · Answer 8 · edited Nov 22 '22 at 19:10

0

I don't have enough rep to add a simple comment. The bash script is great! The edge case I ran into is one where multiple spaces exist, such as a date stamp:

bob/was/here/time 2021-08-31 01:52:51

${line% *} only removes the time while including the date with the topic. Adding a second % to ${line%% *} makes the substitution greedy, ensuring that only the topic is returned.

edited Nov 22 '22 at 19:10

apollosoftware.org

12,161
4
48
69

answered Sep 05 '21 at 21:12

Kenny

125
6

score 0 · Answer 9 · edited Nov 22 '22 at 18:59

0

If you´re on a remote computer you can do this.

mosquitto_sub -h HOST -t 'TOPIC' --remove-retained --retained-only -u USER -P PASSWORD

It works for me !

edited Nov 22 '22 at 18:59

apollosoftware.org

12,161
4
48
69

answered Sep 14 '22 at 14:12

hemerson allan

1

3

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Sep 19 '22 at 06:41

score 0 · Answer 10 · answered Mar 04 '23 at 10:52

From the mosquitto_sub man page

Example 1. Remove all retained messages on the server, assuming we have access to do so, and then exit:
mosquitto_sub -t '#' --remove-retained --retained-only

As a function:

clmq(){ mosquitto_sub -t '#' --remove-retained --retained-only -d -W ${4:-1} -h ${1:-localhost} -u ${2} -P ${3}; }

Example usage:

clmq myhostoripaddr myusername mypassword mytimeout

Note that the option -W 1 causes mosquitto_sub to timeout 1 second after connecting to the broker (so that you don't have to press Ctrl+C to kill the mosquitto_sub process).

If you have a lot of retained messages to clear, you may want to remove the -W 1 option and kill mosquitto_sub manually (once it has finished publishing null messages to all topics which previously had a retained message).

How to clear ALL retained mqtt messages from Mosquitto?

10 Answers10

for powershell users

get retained messages

delete retained messages