Row-based InputStream filtering

Question

I have a large file which I have connected to via a Java InputStream and I will later provide the InputStream to a function as a parameter.

InputStream inputStream = new FileInputStream( ... );

But I want my InputStream to be reduced / filtered to a subset of the row; my data is in the form:

X,Y

X,Y

X,Y

I want the InputStream to include only the second element in this csv file i.e. Y- I want row-based filtering on my stream.

I want to preserve memory when I do such transformations.

Maybe this representation will better explain my requirement:

inputStream -(filter on row)-> filteredInputStream

Then I will pass the filteredInputStream to my function as parameter.

What is the best practice way to do this? Shall I connect one stream to another to perform such filtering?

Asking for libraries is off-topic. Just google for "Java CSV parser". — JB Nizet, Mar 13 '18 at 08:00
I have changed to indicate that I am looking for a best practice approach for help with my problem. I don't see how your Google search term is helpful. — Zeruno, Mar 13 '18 at 08:07
I assume the InputStream gets converted to a Java 8 Stream/List ? This is missing from your question — rjdkolb, Mar 13 '18 at 08:54

score 0 · Answer 1 · answered Mar 13 '18 at 08:42

0

You could try to filter the InputStream by providing your own implementation of a FilterInputStream. In it you can overwrite the read methods so that you can parse the read results and only return the data you want to pass to the next method.

Or, alternatively, you could use a CSV-Parser e.g. the one of apache commons and only pass the Y elements to the sub method. This should be way more easy, but we all don't know how the remaining of your code looks like, so this is only an assumption.

answered Mar 13 '18 at 08:42

femangl

36
5

I am using a library which only takes an inputStream as a parameter to its function. So I want to pass to it an inputStream which is filtered. Can I do this with CSVParser? – Zeruno Mar 13 '18 at 08:46
Not really. Ok, that was what I meant with "only an assumption" if it is easier. So you can try writing your own FilteredInputStream or you parse the elements with a CSV Parser and create an own InputStream out of the result. – femangl Mar 13 '18 at 08:52
e.g. with a ByteArrayInputStream. Seen [here](https://stackoverflow.com/questions/2091454/byte-to-inputstream-or-outputstream) – femangl Mar 13 '18 at 08:55

score 0 · Answer 2 · answered May 15 '18 at 03:57

You're talkings about streaming a file, so I see it's best to use Files.line() stream

// Use try-with-resource to auto close stream
try (Stream<String> lines = Files.lines(Path.getName("your/path"))) { 
    List<String> stringYs = 
            s.map(l -> l.split(","))
             .filter(a -> a.length >= 2) // You may ensure that the string has two parts
             .map(a -> a[1]) // Get the second part, which is "Y"
             .collect(toList());
}

Row-based InputStream filtering

2 Answers2