0
private List<String> processLinesWithLogFilter(List<String> lines) {
    return lines.stream()
        .map(logFilter::filter)
        .filter(Optional::isPresent)
        .map(Optional::get)
        .collect(Collectors.toList());

List<String> lines is a huge list which contains from 300,000 to 1,000,000. My purpose is to filter out the huge log file and be selective on what I would like to be included.

The above code is what is causing high CPU utilization in the java application. I would like to reduce it by any method. I do not mind it taking longer than expected as long as CPU utilization is not high.

This is what logFilter::filter method contains

public Optional<String> filter(String log) {
return extractor.extract(log).filter(filter::isServiceLog).map(printer::printer);

extract method grabs the log file using StringBuilder, filter method uses Filter object using Matchers.

Is there a way to lower the CPU Utilization?

I tried using batch process which divides up the work. It did take longer than the original process but CPU Utilization did not get better.

Young Kim
  • 3
  • 2
  • One thing that occurs to me is that you could use `flatMap` (or, in newer versions of Java, `mapMulti`), and only produce lines that you wanted to include, leaving out all the `Optional` stuff, but that would only reduce the work a bit, not the CPU utilization. – David Conrad Jun 08 '23 at 16:03
  • 2
    The first step is to use a profiling tool to find out *why* it is consuming so much CPU or where the time is spent. Then, you can use this information to find out whether you can reduce the CPU consumption. But it’s striking that you apparently read the entire file into the memory first before processing, instead of using, e.g. `Files.lines` to get a `Stream` in the first place and process the lines while reading. – Holger Jun 08 '23 at 16:49
  • Thank you everyone who contributed to this post. Martheen That post really helped @Holger Thank you for that! I didn't know that was a bad practice. "Reading entire file into memory first before processing" wasn't something I thought about before! – Young Kim Jun 09 '23 at 14:12

0 Answers0