Executorservice exception handling in java

Question

I am using executor service feature of Java. I want to understand the design perspective.

If something goes wrong in one of the batch what will be best approach to handle it?

I am creating fixed thread pool as,

ExecutorService pool = Executors.newFixedThreadPool(10);

Also I am using invokeall() to invoke all callable which is returning future object.

Here is my scenario -

I have 1000 records coming from xml-file and I wanted to save into DB. I created batch of 10, each batch containing 100 records.
Batches started processing(say batch1, batch2, batch3... batch10) and lets say one of batch(batch7) came across error for a particular record while parsing the record from xml and it could not save into DB.
- So my question is how I can handle this situation ?
- How I can get/store failed batch information (batch7 above) ?
- I mean, if there is any error in any of batch should i stop all other batches ?
- Or where i can store information for failed batch and how I can take it for further processing once error corrected ?

You are not "implementing executor service feature of Java." This feature was implemented a long time ago by the creators of Java, and you are now just trying to use it. — Mike Nakis, Dec 05 '15 at 10:35
How are you submitting tasks? With execute or submit method? Have a look at this question:http://stackoverflow.com/questions/34006882/schedule-exceptions-in-java/34011587#34011587 — Ravindra babu, Dec 05 '15 at 11:41
This question is excessively broad. Stack Overflow works best for specific programming issues rather than for general design discussions which can depend on a great many things. Signs that your question is suboptimal include: it contains no code, it contains a number of very broad-based questions, and there are no details of your particular use case. — scottb, Dec 06 '15 at 18:38

score 0 · Answer 1 · answered Dec 05 '15 at 10:34

You should use CompletableFuture to do this

Use CompletableFuture.runAsync( ) to start a process asynchronous, it returns a future. On this future, you can use thenAccept(..) or thenRun(..) methods to do something when process is complete.

There is also a method, exceptionally(..) to do something when an exception is thrown.

By default, it uses a default executor service to do this async, but you can use your own if necessary.

Nandish A · Accepted Answer · 2015-12-06T14:30:58.703

0

The handler that has the logic to process the records should have an variable that stores the batch number.

The handler ideally should have a finite retry logic for few set of database errors.

Once the retry counts exhausts, it warrants a human intervention and it should exit throwing exceptions and the batch number . The executor should ideally should call shutDown . If your logic demands to stop the process immediately , then you should call shutDownNow . Ideally your design should be resistive to such failures and let other batches continue its work even if one fails. Hope it helped you

edited Dec 06 '15 at 14:30

answered Dec 05 '15 at 10:36

Nandish A

1,645
5
26
41

Thanks Nandish...Yes storing the batch number..may be in Db... in some kind of log table and later pick it next batch..looks good idea... – kanishk sharma Dec 06 '15 at 04:36
@kanishksharma Glad it has heled you. Would be great if you can accept the answer by clicking the tick mark so that it helps others too and I get some points – Nandish A Dec 06 '15 at 14:32

score 0 · Answer 3 · answered Dec 06 '15 at 18:33

So my question is how I can handle this situation ?

It all depends on your requirement.

How I can get/store failed batch information (batch7 above) ?

You can store it either in a file or database.

I mean, if there is any error in any of batch should i stop all other batches ?

This depends on your business use case. If you have requirement to stop batch processing even with single batch failure, you have to stop next batches. Otherwise you can continue with next set of batches.

Or where i can store information for failed batch and how I can take it for further processing once error corrected ?

This also depends on your requirement & design. You may have to inform the source about problematic XML file so that they can correct the file and sent it back to you. Once you receive the new copy, you have to push new file for processing. It can be manual or automated which depends on your design.

Executorservice exception handling in java

3 Answers3