Loop arraylist in batches

Question

I want to iterate through an ArrayList in small batch sizes.

For example, if the ArrayList size is 75 and the batch size is 10, I want it to process records 0-10, then 10-20, then 20-30, etc.

I tried this, but it did not work:

int batchSize = 10;
int start = 0;
int end = batchSize;

for(int counter = start ; counter < end ; counter ++)
{
    if (start > list.size())
    {
        System.out.println("breaking");
        break;
    }

    System.out.println("counter   " + counter);
    start = start + batchSize;
    end = end + batchSize;
}

The counter variable did not have correct values. The output was: counter 0 counter 1 counter 2 counter 3 counter 4 counter 5 counter 6 breaking — Anand03, May 28 '14 at 19:43
How does that differ from what you were expecting to see? We can't read your mind, nor do we know what data you have in `list`. — azurefrog, May 28 '14 at 19:44
The arraylist I created contains a random number of String elements [a total of 63 elements]. The counter value in the first iteration is 0 which is correct. But I assume the values should be 10, 20 etc in the subsequent iterations. — Anand03, May 28 '14 at 19:49

score 13 · Answer 1 · edited Oct 28 '19 at 19:10

13

What you need is: Lists.partition(java.util.List, int) from Google Guava

Example:

final List<String> listToBatch = new ArrayList<>();
final List<List<String>> batch = Lists.partition(listToBatch, 10);
for (List<String> list : batch) {
  // Add your code here
}

edited Oct 28 '19 at 19:10

rifaco

746
6
9

answered Nov 19 '15 at 09:12

Tanmay Baid

459
4
19

1

Very comfortable and should be the preferred solution IMHO, especially when data is already present as list. – Steffen Harbich May 13 '16 at 09:12

score 8 · Accepted Answer · edited Jun 08 '17 at 21:22

8

You do it like remainder from batch size and list size to find count.

int batchSize = 10;
int start = 0;
int end = batchSize;

int count = list.size() / batchSize;
int remainder = list.size() % batchSize;
int counter = 0;
for(int i = 0 ; i < count ; i ++)
{
    System.out.println("counter   " + counter);
    for(int counter = start ; counter < end ; counter ++)
    {
        //access array as a[counter]
    }
    start = start + batchSize;
    end = end + batchSize;
}

if(remainder != 0)
{
    end = end - batchSize + remainder;
    for(int counter = start ; counter < end ; counter ++)
    {
       //access array as a[counter]
    }
}

edited Jun 08 '17 at 21:22

MultiplyByZer0

6,302
3
32
48

answered May 28 '14 at 19:50

Karibasappa G C

2,686
1
18
27

Thanks for the inputs, I see the following output for the above: counter 0 counter 1 counter 2 counter 3 counter 4 counter 5..... However, I want to modify the logic is such a way that I can process the arraylist indices in groups of 10 in each iteration. For first iteration, counter starts with 0, for next iteration counter starts with 10 and so on – Anand03 May 28 '14 at 19:59
use like edited answer – Karibasappa G C May 28 '14 at 20:05
Thanks again. The above logic works and i can see the counter variable correctly. Can you please help in modifying the above logic in such a way that: 1) the number of elements processed in each iteration is equal to batch size. 2) I can use the for loop counter variable 'i' to access corresponding arraylist elements in each iteration [meaning: 0 to 9 in first, 10 to 19 in second and so on]. – Anand03 May 28 '14 at 20:27
i have done code modification to your requirement...inner loop takes care of reading list batch wise..outer loop takes care of iterating number of times the batch needs to run..any doubts ask me...vote up and accept answer if satisfied .. – Karibasappa G C May 28 '14 at 20:42
Thanks a lot again for your help. One small doubt. Can we modify the above logic to get rid of the outer loop? If we can put some condition like: if (counter > list.size()) {break;} or any other appropriate logic. – Anand03 May 28 '14 at 20:47
see i dont know why you want to break outer loop..in my above code, for loop iterates only as many times as required...and inner loop iterates ten times for each outer iteration to process list batch wise...example if we have list size 75, then my outer loop iterates 7 times(75/10) and then it comes out automatically...then i am checking reminder and if its > 0 then processing it again...tell me which part you have not understood..or what else you need.. – Karibasappa G C May 29 '14 at 03:32

score 1 · Answer 3 · answered Jul 29 '15 at 19:02

int start = 0;
int end=updateBatchSize;
List finalList = null;
 try {
        while(end < sampleList.size()){
            if(end==sampleList.size()){
                break;
            }
            finalList = sampleList.subList(Math.max(0,start),Math.min(sampleList.size(),end));
            start=Math.max(0,start+updateBatchSize);
            end=Math.min(sampleList.size(),end+updateBatchSize);
        }

Loop arraylist in batches

3 Answers3