Start process queue from where I was?

Question

I have started playing with chronicle and I wrote a process that basically listen to the chronicle queue and do something with it. So every time it restarts, it will reprocess all the events in the file... I wonder what's the best way to do it such that I can persist where I have processed and start from there instead? Like how would you recommend us persist the "index" and is there a way to fast forward?

Answer 1

The usual way to do this is to store the index for each listener (you might write this to shared memory, or to disk, depending on your latency requirements).

There is also a feature called MessageHistory, which can be used when chaining queue processors together. For example, if you were writing to an output queue from one of your listeners, the MessageHistory would automatically store the current index from the source queue in the output queue. Using this method, recovery is a simple procedure of moving to the end of the output queue, reading the source index, then moving to the correct place in the source queue to continue processing.

In terms of 'fast-forwarding', you can use the ExcerptTailer.moveToIndex() method to skip past any messages that you have processed previously.