0

I have spring remote chunking working over JMS. my configurations are as below.

Gateway wait time out = 5000ms, commit interval=10, the number records it can read per read is 1000.

concurrent consumers and max concurrent consumers are 20 and 30 respectively.

Throttle limit is 5.

As per the above configuration, readers reads 70k records and stops, waits for slaves to processes the items read,

Question is why does it stop at 70K when total records needs to be read is 225k?

Changing the commit interval to 2 and numbers records to read per read to 500, changed the total read to 14k, any idea?

The job status remains STARTED

Thanks for the help in advance

user1224036
  • 988
  • 1
  • 15
  • 34
  • Is the queue full? Remote chunking is very heavy on communication between master and slave (you can read about the differences between remote partitioning and remote chunking here: http://stackoverflow.com/questions/20323229/difference-between-spring-batch-remote-chunking-and-remote-partitioning/20342308#20342308). Are you getting any stack traces? – Michael Minella Aug 25 '14 at 14:37
  • No stack traces, the question how its STOP at 70K records and why? – user1224036 Aug 25 '14 at 14:54
  • The job is actually marked with the status `STOPPED` or is it just hung? Configuration and logs would be helpful... – Michael Minella Aug 25 '14 at 16:27

1 Answers1

-2

I had a similar issue when using ActiveMQ for remote chunk. However using kafka there were no issues.

Aamir
  • 16,329
  • 10
  • 59
  • 65
Aravinth Chandrasekaran
  • 1
  • 1