Java: The elegant way to add a line number to a file with lambda

Question

I get used to using lambda to parse file line by line(much neater than bufferedReader.readLine()) for a long time. But today I faced a problem: add a line number to each line.

It need a counter, but the variable in lambda should be effectively final. Finally, I hacked it with an int array.

Code:

public static void main(String[] args) {
    int[] counter = new int[1];
    counter[0] = 0;
    try (Stream<String> lines = Files.lines(Paths.get("/tmp/timeline.txt"), Charset.defaultCharset())) {
        lines.limit(10).forEachOrdered(line -> {
            line = line.trim();
            counter[0] ++;
            System.out.println("Line " + counter[0] + ": " + line);
        });
    } catch (IOException e) {
        e.printStackTrace();
    }
}

Output:

Line 1: p 5714026 wEkQ
Line 2: v 8235889
Line 3: v 6534726
...

My question is, how to avoid my hack and solve that problem elegantly?

does the use of `limit()` imply that you know the number of lines in advance? If so you could use an `IntStream.range()` and then use `mapToObject()` to combine successive ints with successive lines from a `BufferedReader` or stream iterator — Hank D, Apr 07 '16 at 20:19
@HankD No, I don't. Use `limit()` here is for testing my code... — Sayakiss, Apr 08 '16 at 00:53

Holger · Accepted Answer · 2016-04-07T16:42:09.970

There is no elegant functional solution to a non-functional task. The first you may consider, is just resorting to an ordinary anonymous inner class:

String path = "/tmp/timeline.txt";
try(Stream<String> lines = Files.lines(Paths.get(path), Charset.defaultCharset())) {
    lines.limit(10).forEachOrdered(new Consumer<String>() {
        int counter = 0;
        public void accept(String line) {
            System.out.println("Line " + counter++ + ": " + line.trim());
        }
    });
} catch (IOException e) {
    e.printStackTrace();
}

The advantage is that it doesn’t pretend to be functional where it isn’t and the scope of the counter variable has the smallest scope needed for this task.

If you are going to do more than just-printing these numbered lines and need a solution compatible with all stream operations, re-implementing the stream source is a straight-forward solution:

static Stream<String> numberedLines(Path path, Charset cs) throws IOException {
    BufferedReader br = Files.newBufferedReader(path, cs);
    return StreamSupport.stream(new Spliterators.AbstractSpliterator<String>(
            Long.MAX_VALUE, Spliterator.ORDERED|Spliterator.NONNULL) {
        int counter;
        public boolean tryAdvance(Consumer<? super String> action) {
            String line;
            try {
                line = br.readLine();
                if(line==null) return false;
                action.accept("Line " + counter++ + ": " + line.trim());
                return true;
            } catch (IOException ex) {
                throw new UncheckedIOException(ex);
            }
        }
    }, true).onClose(()->{ try { br.close(); }
        catch (IOException ex) { throw new UncheckedIOException(ex); }
    });
}

Of course, this isn’t as simple as a single lambda expression but using this reusable method, you can use all stream operations without problems, e.g.

String path = "/tmp/timeline.txt";
try(Stream<String> lines = numberedLines(Paths.get(path), Charset.defaultCharset())) {
    lines.skip(10).limit(10).forEachOrdered(System.out::println);
} catch(IOException e) {
    e.printStackTrace();
}

score 2 · Answer 2 · answered Apr 07 '16 at 14:13

You might use an AtomicInteger¹ like

AtomicInteger ai = new AtomicInteger();
// ...
lines.limit(10).forEachOrdered(line -> {
    System.out.printf("Line %d: %s%n", ai.incrementAndGet(), line.trim());
});

^{1_{And I would prefer formatted IO with printf to using String concatenation.}}

score 2 · Answer 3 · answered Apr 07 '16 at 14:23

I would implement a Function to number the lines :

public static class LineNumberer implements Function<String,String> {
    private int lineCount;
    public lineNumberer() { lineCount = 0; }
    public String apply(String in) {
        return String.format("%d %s", lineCount++, in);
    }
}


public static void main (String[] args) throws java.lang.Exception
{
    Files.lines(Paths.get("/tmp/timeline.txt")).map(new LineNumberer()).forEach(System.out::println);
}

fps · Answer 4 · 2016-04-09T03:44:07.927

0

If you have a class that holds a pair of values:

public final class Tuple2<A, B> {

    private final A $1;

    private final B $2;

    public Tuple2(A $1, B $2) {
        this.$1 = $1;
        this.$2 = $2;
    }

    public A $1() {
        return $1;
    }

    public B $2() {
        return $2;
    }

    // TODO hashCode equals toString
}

And these methods:

public static <T> Stream<T> streamOf(Iterator<T> iterator) {
    return StreamSupport.stream(
            Spliterators.spliteratorUnknownSize(
                    iterator,
                    Spliterator.ORDERED),
            false);
}

public static <T> Stream<Tuple2<T, Long>> withIndex(
    Stream<T> stream, int startIndex) {

    Iterator<T> it = stream.iterator();
    return streamOf(new Iterator<Tuple2<T, Long>>() {

        private long index = startIndex;

        @Override
        public boolean hasNext() {
            return it.hasNext();
        }

        @Override
        public Tuple2<T, Long> next() {
            return new Tuple2<>(it.next(), index++);
        }
    });
}

which create a stream of pairs, with one element being the original stream's element and the other one being the index, then you could easily solve your problem as follows:

Stream<String> originalStream = lines.limit(10).map(String::trim);

withIndex(originalStream, 1)
    .forEachOrdered(t -> System.out.printf("Line %d: %s%n", t.$2(), t.$1());

NOTE: This only works with sequential streams, which is the case.

edited Apr 09 '16 at 03:44

answered Apr 07 '16 at 18:54

fps

33,623
8
55
110

Your solution lacks the `streamOf` method. Since all JRE provided ways to construct a stream from an iterator work by wrapping it in a spliterator, I discourage from using the `Iterator` detour as you can do the same by operating directly with `Spliterator`, in most cases even simpler. And as a side note, this *does* work with parallel streams as the Stream framework is smart enough to understand that the iterator itself can not be queried concurrently. It’s just that the resulting parallel performance won’t be impressive… – Holger Apr 08 '16 at 08:41
@Holger I don't get the streamOf part... And yes, in general, it would be much better to use a spliterator. However, this is to process lines sequentially, so no need for a spliterator. – fps Apr 08 '16 at 11:49
There is no `streamOf` method in the standard API, so you have to either, tell where it comes from or provide the code for it. – Holger Apr 08 '16 at 12:18
I’m not recommending the `Spliterator` for parallel capabilities. I recommend it, because it’s easier to implement (one method instead of two) and because it’s the canonical backend of the Stream framework, thus it’s more efficient to implement a spliterator atop the stream’s spliterator, instead of letting the stream wrap its spliterator in an iterator to implement an iterator atop it which will get wrapped into a spliterator for the new stream… – Holger Apr 08 '16 at 12:24

score 0 · Answer 5 · answered Nov 18 '21 at 18:25

Jast use zip. What is zip (functional programming?)

import com.google.common.collect.Streams;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.stream.Stream;

class Pair<F, S> {
    public final F first;
    public final S second;
    public Pair(F first, S second) {
      this.first = first;
      this.second = second;
    }
    public String toString(){
        return second.toString()+": "+first;
    }
}

public class StreamExample {
    static public void show(String fileName){
        Stream<Integer> streamInt = Stream.iterate(1, i -> i + 1);
        try (Stream<String> streamStr = Files.lines(Paths.get(fileName))) {
        Streams.zip(
                streamStr,streamInt,
                (firstStr, secondStr) ->new Pair(firstStr,secondStr))
                .limit(10)
                .forEach(System.out::println);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Java: The elegant way to add a line number to a file with lambda

5 Answers5

Linked