How most effective add some info to list with multiple threads in Java?

Question

I have some task to aggregate some info from multiple links in most effective way, using multithreading. The links are at some array. By now, i have something like this solution:

Arrays.stream(link).parallel().forEach(link -> {
        try {
            String result = doSomeJobWithLink(link);
            System.out.println(result);
        } catch (IOException e) {
            e.printStackTrace();
        }
    });

And it works pretty good (job has done for 2 secs).

But I wanna not to print result in my try block, but collect results in some list (or other collection), so i did it this way:

List<String> resultList = Collections.synchronizedList(new ArrayList<>());
Arrays.stream(link).parallel().forEach(link -> {
        try {
            String result = doSomeJobWithLink(link);
            resultList.add(result);
        } catch (IOException e) {
            e.printStackTrace();
        }
    });
resultList.forEach(System.out::println);

But it's took about 5-8 sec's instead of two. Can I speed it up someway?

Try setting the initial capacity of the resultList to the size of the input array. — David G., Dec 30 '19 at 09:47

score 2 · Answer 1 · answered Dec 30 '19 at 10:11

When you do, Collections.synchronizedList(new ArrayList<>()), you put a synchronized on the whole list i.e any operation on the list share the same mutex even reads, this has a high performance penalty and is the limiting factor.

A better way is to just collect to a normal list, Collector guarantees an unordered concurrent reduction.

For concurrent collectors, an implementation is free to (but not required to) implement reduction concurrently. A concurrent reduction is one where the accumulator function is called concurrently from multiple threads, using the same concurrently-modifiable result container, rather than keeping the result isolated during accumulation. A concurrent reduction should only be applied if the collector has the Collector.Characteristics.UNORDERED characteristics or if the originating data is unordered.

So the below should significantly improve performance,

List<String> resultList = Arrays.stream(link).parallel().map(e -> {
            try {
                return doSomeJobWithLink(e);
            } catch (IOException ex) {
                ex.printStackTrace();
                return null;
            }
            return result;
        }).filter(Objects::nonNull).collect(Collectors.toList());

Though it is not recommended to swallow exceptions, unless that is unavoidable.

Sumit Singh · Accepted Answer · 2019-12-30T10:02:12.197

Use below code:

List<String> resultList = Arrays.stream(link).parallel().map(v -> doSomeJobWithLink(v)).collect(Collectors.toList());

Normally we avoid try-catch in stream pipe but if you must have to catch exception please read Exception Handling in Java Streams

And don't use parallel just because you can, because of additional overhead your job will take more time then without parallel

see Should I always use a parallel stream when possible?

score 1 · Answer 3 · answered Dec 30 '19 at 09:50

Not sure about it if the below code will improve any performance or not but I think it will be a cleaner approach to solve your problem.

List<String> resultList = Arrays.stream(link).parallel().map(e -> {
            String result = null;
            try {
                result = doSomeJobWithLink(e);
            } catch (IOException ex) {
                ex.printStackTrace();
                return null;
            }
            return result;
        }).filter(e -> e != null).collect(Collectors.toList());

Returning null is always the first step towards a NPE. Dont do that. Either throw a custom exception, or return an empty list. — GhostCat, Dec 30 '19 at 10:05

How most effective add some info to list with multiple threads in Java?

3 Answers3