0

I have some JSR-352 batch jobs that run quite well - despite their runtime. Now I am thinking of distributing the work across several threads.

How is this pattern supported by JSR-352?

Edit: Now that I know the keywords to search for I can make out more resources on this problem:

I want to create a partitioned batchlet, and the partitions must be calculated at runtime. The idea is to split the processing of all records into a predefined maximum number of partitions, or a number of partitions with maximum size.

Queeg
  • 7,748
  • 1
  • 16
  • 42
  • 1
    See https://docs.spring.io/spring-batch/docs/current/reference/html/jsr-352.html, section Partitioning. – pringi Feb 28 '22 at 18:26
  • Thank you, Partitioning seems the answer. And I do not understand why this question would get downvoted without even a comment. – Queeg Mar 01 '22 at 06:50

1 Answers1

1

Here is another example of using partitioned chunk processing in JBeret. Partitioned batchlet should be similar in terms of job configuration. But partitioned batchlet is not common compared to partitioned chunk steps, because batchlet is mainly a task-oriented step, which is not easy to split into multiple partitions. So you may want to check if batchlet step is really suitable.

JBeret project test-apps module contains some examples with partition configuration. For example, cdiScopes tests use batchlet with partitions. Throttle tests use chunk with partitions.

JBeret test-apps/chunkPartition test module contains a job xml that uses mapper for dynamic partition.

cheng
  • 1,076
  • 6
  • 6
  • In my case, the batch needs to read records, do some processing and write them back plus maybe create additional information. So far I thought the reader/writer pattern does not apply. Hence I have a batchlet. But there is no reason not to parallelize. Thank you for the example, I will take a look. – Queeg Mar 08 '22 at 19:23
  • The link you posted does not work for me – Queeg Mar 09 '22 at 07:48
  • Sorry for the bad link. Now fixed. – cheng Jan 26 '23 at 14:44
  • Thank you for the fixed link. Now I am wondering where in that example parallel threads play a role, something I hoped to achieve with partitioning. – Queeg Jan 26 '23 at 21:34
  • I edited my answer above to include some examples of partitioned chunk and partitioned batchlet. – cheng Jan 29 '23 at 23:43
  • The newly referenced examples use partition, thank you. Do you also happen to have one with a dynamically calculated partition? – Queeg Jan 30 '23 at 07:48
  • I just added to the above answer a link to a job xml that uses dynamic partition. – cheng Jan 30 '23 at 17:01
  • How does JBeret handle the ref attributes? I do not see a mapping to class names within the batch xml file. – Queeg Jan 30 '23 at 20:12
  • Please see [Jakarta Batch Spec](https://github.com/eclipse-ee4j/batch-api/blob/master/spec/src/main/asciidoc/batch_runtime_spec.adoc#batch-artifact-loading) for details of loading batch classes. – cheng Jan 30 '23 at 22:40