How to iterate over every n elements with java?

Question

I have a method named calculate and it takes too long to complete. So I decided to send my info list objects to this method partially. How can I iterate over every n elements?

public static void main(String [] args){
    Map<String, Long> info....;  //my info Map

    //I want to call method like 
    for(int i = 0; i<info.size(); i+=5)
       calculate(info.submap(i,i+5)); 
}


public static boolean calculate(Map<String, Long> info){
    //Some calculations

}

You are just adding and removing the stackframe `info.size()/5` times. — Azodious, Dec 29 '16 at 11:47
I just want to send info objects 5 by 5 to calculate method. I don't get what you mean. — hellzone, Dec 29 '16 at 11:48
@hellzone: i meant that by sending it partially in this way you are not gaining anything in terms of time performance. — Azodious, Dec 29 '16 at 11:51
@Azodious calculate method sends this list to a web service and there is a problem with this webservice. When I sent 100 info, it responses nearly after 1 hour and when I sent 5 info it responses in 1 second. — hellzone, Dec 29 '16 at 11:56
Parallel Stream is a good suggestion then, by Jobin. Check this too: http://stackoverflow.com/questions/20375176/should-i-always-use-a-parallel-stream-when-possible — Azodious, Dec 29 '16 at 12:00

score 0 · Answer 1 · answered Jan 01 '17 at 20:28

You can use following code

class SomeClass {

    private final int BUFFER_SIZE = 5;    

    public static void main(String[] args) {
        Map<String, Long> info = new HashMap<>();

        LongStream.range(0, 30).boxed().forEach(i -> info.put("key" + i, i)); // for test

        IntStream.range(0, info.size() / BUFFER_SIZE)
                .boxed()
                .parallel()
                .map(i -> Arrays.copyOfRange(info.keySet().toArray(), BUFFER_SIZE * i, BUFFER_SIZE * (i + 1)))
                .map(Arrays::asList)
                .map(keys -> info.entrySet().stream()
                        .filter(x -> keys.contains(x.getKey()))
                        .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue)))
                .forEach(SomeClass::calculate);
    }

    public static boolean calculate(Map<String, Long> info) {
        System.out.println("calculation for " + info.toString());
        return true;
    }
}

score 0 · Answer 2 · edited May 23 '17 at 11:45

It sounds like what you want to do is to implement a sort of batch processing for the data represented by your Map<String, Long> info instance. You can then create a generator for these batches as a Stream: This is in a way the inverse of the Stream.flatMap(...) family of methods, but, ironically, there doesn't seem to be any idiomatic functional way of doing this and so you may have to create the batches yourself in an imperative manner — for example:

private static <T> Stream<Stream<T>> createBatchStreams(final Iterator<T> iter, final int maxBatchSize) {
    final Stream.Builder<Stream<T>> resultBuilder = Stream.builder();
    {
        // NOTE: This logic could also be encapsulated in a Collector class
        // in order to make it less imperative style
        Stream.Builder<T> currentBatchBuilder = Stream.builder();
        int currentBatchSize = 0;

        while (iter.hasNext()) {
            final T next = iter.next();
            if (currentBatchSize == maxBatchSize) {
                resultBuilder.add(currentBatchBuilder.build());
                // Start building a new batch
                currentBatchBuilder = Stream.builder();
                currentBatchSize = 0;
            }
            currentBatchBuilder.add(next);
            currentBatchSize++;
        }
        // Check if there is a non-empty Stream to add (e.g. if there was a
        // final batch which was smaller than the others)
        if (currentBatchSize > 0) {
            resultBuilder.add(currentBatchBuilder.build());
        }
    }
    return resultBuilder.build();
}

Using this method, you can then create a generator of batches of your map data, which can then be fed to your calculate(...) function (albeit with a slightly different signature):

public static void main(final String[] args) {
    final Map<String, Long> info = LongStream.range(0, 10).boxed()
            .collect(Collectors.toMap(value -> "key" + value, Function.identity())); // Test data
    final Stream<Stream<Entry<String, Long>>> batches = createBatchStreams(info.entrySet().iterator(), 5);
    batches.forEach(batch -> {
        calculate(batch);
        // Do some other stuff after processing each batch
    });
}

private static boolean calculate(final Stream<Entry<String, Long>> info) {
    // Some calculations
}

score 0 · Answer 3 · answered Feb 24 '23 at 07:52

You loop n-times over the collection and take the range of items you need.

         final int AMOUNT_OF_ITEMS_IN_BATCH = 200;
        List<Object> objects = new ArrayList<>();//TODO YOUR LIST WITH OBJECTS TO SPLIT UP AND PROCESS IN ITERATIONS
        int loops = IntMath.divide(objects.size(), AMOUNT_OF_ITEMS_IN_BATCH, RoundingMode.CEILING);

        for (int i = 0; i < loops; i++) {
            Set<Object> nextBatch = objects.stream()
                    .skip(i * AMOUNT_OF_ITEMS_IN_BATCH)
                    .limit(AMOUNT_OF_ITEMS_IN_BATCH)
                    .collect(Collectors.toSet());

           //TODO WHAT YOU WANT TO DO WITH TE SUBLIST 
        }

How to iterate over every n elements with java?

3 Answers3