0

I have seen the AmqpJobSample

http://docs.spring.io/spring-batch/spring-batch-samples/#AmqpJobFunctionalTests

It uses AmqpItemReader to read from rabbitmq message queue, processes it using message handler and then writes back to queue using AmqpItemWriter.

My use case is to, read items from listItemReader which I have implemented, then sends items to messaging queue which will be processed by consumers running on different servers (rabbitmq cluster) and those consumers will respond back whether the iterm are processed successfully or not for each item.

Then I will read it from message queue (in itemWriter) and if check the statuses of item processing and return the status from itemWriter if its successfull or will throw exception if it is failed.

My application is in PHP Symfony framework but I am introducing spring batch for batch process which is currently in shell script.

I am using spring batch to produce messages but my consumers are in PHP which runs on different servers. I just want to pass ids as a messages and want to process ids using PHP consumers on different servers.

How can I do remote chunking or partitioning?

http://docs.spring.io/spring-batch/reference/html/scalability.html

Are there any samples for remote chunking and partitioning using messaging e.g. RabbiyMq ?

How can I this using AmqpItemReader and AmqpItemWriter ? Do I need to use Request/Reply ? Can I set reply-to queue ? Whats the best way to do this ?

vishal
  • 3,993
  • 14
  • 59
  • 102

1 Answers1

0

It sounds like you should be using remote chunking instead.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Actully I am bit confused how is this different from remote chunking? Isnt this remote chunking ? – vishal Apr 16 '14 at 13:18
  • I am using spring batch to produce messages but my consumers are in PHP (my application is in PHP Symfony but I am introducing spring batch for batch process which is currently in shell script) which runs on different servers. I just want to pass id as a message and want to process ids using PHP consumers on different servers. – vishal Apr 16 '14 at 13:22
  • The standard batch infrastructure (for chunking or partitioning) requires the workers to be batch components. You can do what you want, but you'll need your own plumbing, maybe a custom `ItemProcessor`. – Gary Russell Apr 16 '14 at 13:25
  • +1 for Gary's recommendation of using Remote Chunking. You can read about remote chunking in this related response: http://stackoverflow.com/questions/20323229/difference-between-spring-batch-remote-chunking-and-remote-partitioning/20342308#20342308 – Michael Minella Apr 17 '14 at 14:41
  • @MichaelMinella, I have already had looked at it but not able to understand how to apply it to my use case. There are no samples as well. – vishal Apr 19 '14 at 22:40
  • For samples, check out the unit tests here: https://github.com/spring-projects/spring-batch/tree/master/spring-batch-integration/src/test/resources/org/springframework/batch/integration/chunk – Michael Minella Apr 21 '14 at 14:30