What is the best way to convert a byte array to an IntStream?

Question

Java 8 has java.util.stream.Stream and java.util.stream.IntStream types. java.util.Arrays has a method

IntStream is = Arrays.stream(int[])

but no such method to make an IntStream from a byte[], short[] or char[], widening each element to an int. Is there an idiomatic/preferred way to create an IntStream from a byte[], so I can operate on byte arrays in a functional manner?

I can of course trivially convert the byte[] to int[] manually and use Arrays.stream(int[]), or use IntStream.Builder:

public static IntStream stream(byte[] bytes) {
   IntStream.Builder isb = IntStream.builder();
   for (byte b: bytes) 
       isb.add((int) b); 
   return isb.build();
}

but neither is very functional due to the copying of the source.

There also does not seem to be an easy way to convert an InputStream (or in this case an ByteArrayInputStream) to an IntStream, which would be very useful for processing InputStream functionally. (Glaring omission?)

Is there a more functional way that is efficient and does not copy?

Every `byte` should be considered an `int`? Or every 4 bytes should be considered an `int`? — Sotirios Delimanolis, Jan 07 '15 at 01:56
every byte is widened to an int. for example, I want to calculate a historgram of all the bytes in a file or other byte stream. — djb, Jan 08 '15 at 03:29
Why do you want to read bytes from an InputStream using the stream API? InputStream reads are sequential, it's much faster to read using the old method, since you can't parallel process an ordered stream of individual bytes in any way. You might get a lot of benefit from parallel processing *chunks* of an InputStream... — Steve K, Jan 08 '15 at 22:44
This is basically asking "why add lambdas and streams to Java?" A: So I can write a function that I can apply to any IntStream (not just InputStream), and compose with other functions and map/flatMap/filter etc. and not have to write one way of doing it for a byte[] and another way of doing it for an InputStream and another way for some other source of bytes... — djb, Jan 09 '15 at 02:07

Paul Boddington · Accepted Answer · 2015-01-07T02:12:09.510

25

 byte[] bytes = {2, 6, -2, 1, 7};
 IntStream is = IntStream.range(0, bytes.length).map(i -> bytes[i]);

 ByteArrayInputStream inputStream = new ByteArrayInputStream(bytes);
 IntStream is2 = IntStream.generate(inputStream::read).limit(inputStream.available());

edited Jan 07 '15 at 02:12

answered Jan 07 '15 at 01:58

Paul Boddington

37,127
10
65
116

3

Add a `limit` to the `generate`. Otherwise it returns an infinite `Stream`. – Sotirios Delimanolis Jan 07 '15 at 02:08
I'm voting this as an answer for the byte[] to IntStream portion. However, while inputStream.available() is valid for a ByteArrayInputStream, it is not accurate for all InputStreams, so using it to limit the stream may result in truncated streams if used generally. Is there another resolution to the general InputStream to IntStream problem? – djb Jan 07 '15 at 03:04
@djb I don't know I'm afraid. If you want better answers to the second bit I suggest you unaccept my answer as people are less likely to answer if it's already accepted. I don't mind losing the points. :) – Paul Boddington Jan 07 '15 at 03:08
1

@djb You could read the full `InputStream` into a `ByteArrayInputStream` or `byte[]` and do the other conversion. – Sotirios Delimanolis Jan 07 '15 at 03:11
2

Converting an arbitrary `InputStream` to an `IntStream` simply is a different question. Not only because of the different `available` contract but also because of checked exceptions… – Holger Jan 07 '15 at 11:37

score 10 · Answer 2 · edited May 23 '17 at 10:31

public static IntStream stream(byte[] bytes) {
    ByteBuffer buffer = ByteBuffer.wrap(bytes);
    return IntStream.generate(buffer::get).limit(buffer.remaining());
}

(This can easily be changed to take ints from the ByteBuffer, ie. 4 bytes to the int.)

For InputStream, if you want to consume it eagerly, just read it into a byte[] and use the above. If you want to consume it lazily, you could generate an infinite InputStream using InputStream::read as a Consumer (plus exception handling) and end it when you've reached the end of the stream.

Concerning

but neither is very functional due to the copying of the source

I don't see why that makes it non functional.

Also relevant

For bonus points, use a method reference for buffer.get(): `IntStream.generate(buffer::get).limit(buffer.remaining())` — prunge, Jan 07 '15 at 04:46

What is the best way to convert a byte array to an IntStream?

2 Answers2

Linked