How to subdivide a list into multiple lists while processing in Java stream?

Question

I have the following API call

cloudwatchClient.deleteInsightRules(
    new DeleteInsightRulesRequest().withRuleNames(existingRuleNames));

which fails because

 [exec] Exception in thread "main" com.amazonaws.services.cloudwatch.model.MissingRequiredParameterException: BATCH_REQUEST_TOO_LARGE: Batch size cannot exceed 20 items. (Service: AmazonCloudWatch; Status Code: 400; Error Code: MissingRequiredParameterException; Request ID: 0f820dc2-38b8-11e9-8465-3718764429f1)

Now I understand that I have to make multiple calls to deleteInsight rules for cloudwatch in chunks of 20.

So conceptually I am looking for

existingRuleNames
   .stream().
   .chunkIntoSizeOf(20)          // This is an imaginary method, i need to convert it into something concrete
   .foreach(chunk -> cloudwatchClient.deleteInsightRules(
        new DeleteInsightRulesRequest().withRuleNames(chunk));

Now I cant find anything in the java 8 streams api that allows me to divide and process my lists into chunks. Kind of like the scala grouped functionality Split list into multiple lists with fixed number of elements.

Can anyone help me out with this? Thanks. I can of course use an imperative style and use sublists, but I prefer to avoid that if I can.

Kartik · Answer 1 · 2019-02-25T05:57:54.627

In the forEach below you can do whatever you want to do with the chunk:

//my list of 30 strings that needs to be processed (existingRuleNames in your case):
List<String> list = IntStream.range(0, 30).mapToObj(String::valueOf).collect(Collectors.toList());

AtomicInteger prev = new AtomicInteger(0);
IntStream.range(1, (int) (20 * (Math.ceil(Math.abs(list.size() / 20.0)))))
        .filter(i -> i % 20 == 0 || i == list.size())
        .forEach(i -> {
            List<String> chunk = list.subList(prev.get(), i);
            System.out.println("Processing " + chunk);
            prev.set(i);
        });

Output:

Processing [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
Processing [20, 21, 22, 23, 24, 25, 26, 27, 28, 29]

So your code should look something like:

List<String> existingRuleNames = ...;

AtomicInteger prev = new AtomicInteger(0);
IntStream.range(1, (int) (20 * (Math.ceil(Math.abs(existingRuleNames.size() / 20.0)))))
        .filter(i -> i % 20 == 0 || i == existingRuleNames.size())
        .forEach(i -> {
            List<String> chunk = existingRuleNames.subList(prev.get(), i);
            cloudwatchClient.deleteInsightRules(
                    new DeleteInsightRulesRequest().withRuleNames(chunk));
            prev.set(i);
        });

How to subdivide a list into multiple lists while processing in Java stream?

1 Answers1