0

In activemq 5, each queue had a folder containing its data and messages, everything. Which would mean that, in case of an issue, for example an out of disk space error. Some files would get corrupted before the server crash. In that case, in activemq 5, we would find logs indicating corrupted files, and we could delete the queue folder that was corrupted, resulting in small loss of messages instead of ALL messages.

In artemis, it seems that messages are stored in the same files, independently from the queue they are stored in. Which means if i get an out of disk space error, i might have to delete all my messages.

First, can you confirm the change of behaviour, and secondly, is there a way to recover ? And a bonus, if anyone know why this change happened, I would like to understand.

Artandor
  • 472
  • 6
  • 20

1 Answers1

0

Artemis uses a completely new message journal implementation as compared to 5.x. The same journal is used for all messages. However, it isn't subject to the same corruption problems as you've seen with 5.x. If records from the journal can't be processed then they are simply skipped.

If you get an out of disk space error you should never need to delete all your messages. The journal files themselves are allocated and filled with zeroes to meet their configured size before they are actually used so if you were going to run out of disk space you'd do so during that process before any messages were written to them.

The Artemis journal implementation was written from the ground up for high performance specifically in conjunction with the broker's non-blocking architecture.

Justin Bertram
  • 29,372
  • 4
  • 21
  • 43
  • So what happens when i get an out of disk for example ? Corrupted files are skipped and the broker manage to restart no matter what ? – Artandor Aug 23 '19 at 14:18
  • I think you answered my concerns yeah. Even if i'm still not sure that in case of a crash i would not lose messages here. – Artandor Aug 28 '19 at 09:48
  • 1
    If you have additional questions feel free to ask. Otherwise I recommend you simply perform testing of your own and even take a look at the source code to see how it works. – Justin Bertram Aug 28 '19 at 11:56
  • Are there tools to read/manage/edit those files ? There's a lot of tools to read the kahadb without activemq running – Artandor Aug 28 '19 at 14:45
  • You can read the journal files using the `data print` command that I already mentioned to you are on your [other question](https://stackoverflow.com/questions/57272240/artemis-durable-subscription-message-storage). There are no external tools to edit the files. I'm not sure what you mean by "manage." – Justin Bertram Aug 28 '19 at 14:51
  • By manage i mean delete some messages, change order, ... Considering data print launch an activemq server, if for some reason the server is not able to start, there is currently no way to retrieve the files ? Also, if i wanted to backup some files while doing this, what would i need ? Bindings and journals ? – Artandor Aug 28 '19 at 15:26
  • I'd categorize deleting & changing message order as "editing" which, as I've already indicated, is not supported by any external tools. Printing the data does not "launch an activemq server." How did you come to that conclusion? To get a full backup for a broker you'd want to back-up all the persistent data (i.e. broker.xml as well as the journal, bindings, paging, & large messages directories). – Justin Bertram Aug 28 '19 at 15:40
  • Maybe i jumped to this conclusion too fast when i tried to run the command and i got something like "activemq is already running and has a pid". Thanks for all your answers, have a nice day – Artandor Aug 29 '19 at 08:00
  • 1
    The reason you got that message is that the ActiveMQ Artemis data tools should only be run while the broker is shutdown. – Justin Bertram Aug 29 '19 at 13:14