3

I have a collection of "business-process" documents and a separate collection of "business-process-events", each with a property business-process-id as a reference to the business-process of the event. Both documents are immutable to avoid concurrency issues, at least that is the idea so far.

I need to perform queries and show business-processes in a read-only "grid" with supplemental data based on a lot of logic from the business-process-events that has occured. Business-process-events can be stored out of order, especially

I thought about using map-reduce, but from past experience and available documentation, it will not be possible to do enough complex logic in the reduce.

If it was possible to subscribe to all new business-process-events and update a new index based on all business-process-events for the business-process at that point it would probably be a good solution, but how?

Trygve
  • 2,445
  • 1
  • 19
  • 23
  • 1
    So each "business-process-event" document contains a property with the related "business-process" document ID ? You can always load a related document when querying, and index related documents - maybe you can clarify your question ? – Danielle Aug 20 '23 at 12:44
  • @Danielle. Yes, each "event" document contains a property with the related "process" document ID. Yes, I've thought about indexing by "event" and for each event load the related "process". But if that index is only a map index I will have to do a lot of processing on every query to group "events" by "process-id" – Trygve Aug 20 '23 at 19:25
  • 1
    If your "events" document has a property that is the "process-id" then you can make a simple map-index on the "events" docs, indexing just the "process-id" (not indexing the related document itself) and then you can query the index and get ALL events that match a specific process-id. Isn't that what you want ? – Danielle Aug 22 '23 at 10:21
  • I decided to go somewhat of a different route. See my answer – Trygve Aug 24 '23 at 11:43

1 Answers1

2

Instead of trying to solve this with an index, I dispatch a message (command) on a queue to perform the complex logic based on all the "events". This message is dispatched every time a new "event" occurs. Since the processing every time takes all "events" into consideration each time, the result is idempotent and even though there could be some duplicate processings, that is of no concern as long as the data is complete. We store the result of the complex logic processing in the "business-process" and make a simple map index on that.

We use NServiceBus as a messaging framework, but any kind of queueing can be used.

Trygve
  • 2,445
  • 1
  • 19
  • 23