9

background

suppose i have an inputStream that was originated from the internet of a certain image file.

i wish to get information about the image file and only then to decode it.

it's useful for multiple purposes, such as downsampling and also previewing of information before the image is shown.

the problem

i've tried to mark&reset the inputStream by wrapping the inputStream with a BufferedInputStream , but it didn't work:

inputStream=new BufferedInputStream(inputStream);
inputStream.mark(Integer.MAX_VALUE);
final BitmapFactory.Options options=new BitmapFactory.Options();
options.inJustDecodeBounds=true;
BitmapFactory.decodeStream(inputStream,null,options);
//this works fine. i get the options filled just right.

inputStream.reset();
final Bitmap bitmap=BitmapFactory.decodeStream(inputStream,null,options);
//this returns null

for getting the inputStream out of a url, i use:

public static InputStream getInputStreamFromInternet(final String urlString)
  {
  try
    {
    final URL url=new URL(urlString);
    final HttpURLConnection urlConnection=(HttpURLConnection)url.openConnection();
    final InputStream in=urlConnection.getInputStream();
    return in;
    }
  catch(final Exception e)
    {
    e.printStackTrace();
    }
  return null;
  }

the question

how can i make the code handle the marking an resetting ?

it works perfectly with resources (in fact i didn't even have to create a new BufferedInputStream for this to work) but not with inputStream from the internet...


EDIT:

it seems my code is just fine, sort of...

on some websites (like this one and this one), it fails to decode the image file even after reseting.

if you decode the bitmap (and use inSampleSize) , it can decode it fine (just takes a long time).

now the question is why it happens, and how can i fix it.

android developer
  • 114,585
  • 152
  • 739
  • 1,270

4 Answers4

0

I believe the problem is that the call to mark() with the large value is overwritten by a call to mark(1024). As described in the documentation:

Prior to KITKAT, if is.markSupported() returns true, is.mark(1024) would be called. As of KITKAT, this is no longer the case.

This may be resulting in a reset() fail if reads larger than this value are being done.

hdante
  • 7,685
  • 3
  • 31
  • 36
  • didn't test it on kitkat, but i don't get what the docs say now. if is.mark(1024) isn't called, then what is? in any case, the problem i've reported still can't be handled like what i've written. the question now is: how do you do it correctly? – android developer Jan 19 '14 at 00:03
  • This is the documentation of decodeStream. It means it calls is.mark(1024) internally, overwriting your call. How to do it correctly: reimplement decodeStream without the call to is.mark(1024). Else, you need to generate the input stream twice. – hdante Jan 19 '14 at 02:05
  • How do I do that? Simple copying from the source code won't be enough, since a lot of it is internal and private and I assume I also need the native code (C/C++) in this case... – android developer Jan 19 '14 at 06:26
  • I'm not sure. I noticed that there's excessive encapsulation in the code, which makes it hard or impossible to reuse it (and that's a common theme both in android library and in java projects in general). If I went through this path, I'd fully copy and paste the whole BitmapFactory.java file, removing the problematic parts. I didn't try it this way, though, I went for generating the input stream twice. – hdante Jan 19 '14 at 17:22
  • i think it's better to download the file to the cache and then read from it instead of connecting to the server twice. about the encapsulation, you are correct. i have no idea why they made it so "secure". it's just bitmaps... – android developer Jan 19 '14 at 17:52
0

(Here is a solution for the same problem, but when reading from disk. I didn't realize at first your question was specifically from a network stream.)

The problem with mark & reset in general here is that BitmapFactory.decodeStream() sometimes resets your marks. Thus resetting in order to do the actual read is broken.

But there is a second problem with BufferedInputStream: it can cause the entire image to be buffered in memory along side of where ever you are actually reading it into. Depending on your use case, this can really kill your performance. (Lots of allocation means lots of GC)

There is a really great solution here: https://stackoverflow.com/a/18665678/1366

I modified it slightly for this particular use case to solve the mark & reset problem:

public class MarkableFileInputStream extends FilterInputStream
{
    private static final String TAG = MarkableFileInputStream.class.getSimpleName();

    private FileChannel m_fileChannel;
    private long m_mark = -1;

    public MarkableFileInputStream( FileInputStream fis )
    {
        super( fis );
        m_fileChannel = fis.getChannel();
    }

    @Override
    public boolean markSupported()
    {
        return true;
    }

    @Override
    public synchronized void mark( int readlimit )
    {
        try
        {
            m_mark = m_fileChannel.position();
        }
        catch( IOException ex )
        {
            Log.d( TAG, "Mark failed" );
            m_mark = -1;
        }
    }

    @Override
    public synchronized void reset() throws IOException
    {
        // Reset to beginning if mark has not been called or was reset
        // This is a little bit of custom functionality to solve problems
        // specific to Android's Bitmap decoding, and is slightly non-standard behavior
        if( m_mark == -1 )
        {
            m_fileChannel.position( 0 );
        }
        else
        {
            m_fileChannel.position( m_mark );
            m_mark = -1;
        }
    }
}

This won't allocate any extra memory during reads, and can be reset even if the marks have been cleared.

Community
  • 1
  • 1
Adam
  • 25,966
  • 23
  • 76
  • 87
  • I'm not familiar with "FilterInputStream", but as it takes "FileInputStream" instead of a simple InputStream, this means that I need to give it a file that exists on the device, no? If not, did you check that it works? – android developer Jun 03 '14 at 20:59
  • Correct, in my use case I take the network stream and save it to a disk cache. Then my actual image loader always uses a FileInputStream to read from the cache. FilterInputStream is simply a wrapper that allows you to transform an InputStream as it is read. FilterInputStream can take any InputStream, however MarkableFileInputStream is predicated on it being a FileInputStream due to the use of a FileChannel to do the mark & reset functionality. – Adam Jun 04 '14 at 21:27
  • I'd recommend using DiskLruCache if you're interested in caching the images. https://github.com/JakeWharton/DiskLruCache/ It can really help reduce redundant network traffic and load times one you've got your data the first time. – Adam Jun 04 '14 at 21:31
  • The thing is that the question was how to avoid using the disk and do it all using the inputStream, so this isn't much the answer (though it might be correct). In the end though, I decided to use the disk as it's easier and might be better anyway. What is the difference between using what you did and simply using BufferedInputStream? – android developer Jun 05 '14 at 07:42
  • I put it in the answer, but BufferedInputStream can cause, at worst, an entire second copy of the data to be resident in memory while you're reading it. That's how BufferedInputStream provides the functionality it provides, by throwing the data into an intermediate Buffer. – Adam Jun 05 '14 at 17:18
  • I see, so it's better than using BufferedInputStream. But if you already have the file itself, you could just call BitmapFactory.decode twice : once with inJustDecodeBounds=true (to get the information of the image), and the second set it to false, to do the real downsampling. – android developer Jun 05 '14 at 22:18
-1

whether you can mark / reset a stream depends on the implementation of the stream. those are optional operations and aren't typically supported. your options are to read the stream into a buffer and then read from that stream 2x, or just make the network connection 2x.

the easiest thing is probably to write into a ByteArrayOutputStream,

ByteArrayOutputStream baos = new ByteArrayOutputStream();
int count;
byte[] b = new byte[...];
while ((count = input.read(b) != -1) [
  baos.write(b, 0, count);
}

now either use the result of baos.toByteArray() directly, or create a ByteArrayInputStream and use that repeatedly, calling reset() after consuming it each time.

ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());

that might sound silly, but there's no magic. you either buffer the data in memory, or you read it 2x from the source. if the stream did support mark / reset, it'd have to do the same thing in it's implementation.

Jeffrey Blattman
  • 22,176
  • 9
  • 79
  • 134
  • can't BufferedInputStream just hold the bytes that i will return to soon ? i don't wish to read the entire file or create a new connection since it misses the whole point - i already have the inputStream and its data is also available... – android developer Jul 21 '13 at 16:42
  • it depend on the implementation of the stream that was used to construct the `BufferedInputStream`. does it support `reset()`? – Jeffrey Blattman Jul 21 '13 at 16:43
  • it should, no? anyway, suppose i use your method, doesn't it mean i download the entire bytes of the file? – android developer Jul 21 '13 at 16:45
  • no, it should not. what you have to understand is that a stream is, well, a stream, as opposed to a buffer. it does not inherently buffer or cache the data after it is read. that's the whole point of a stream. and yes, it means you have to download and store in memory the entire bytes of the file, or you have to read it 2x from the source. those are your options. i'm curious where you think the data would come from. it's either in memory, or read from the source again. there's no magic here. – Jeffrey Blattman Jul 21 '13 at 16:46
  • is there an inputStream class that caches the bytes, so that i could read some bytes, and then return back to some position ? i could even tell it how much bytes to support to go back if needed... – android developer Jul 21 '13 at 16:47
  • yes, there's a class. it's called a `byte[]`, or if you need the data available in a stream form, a `ByteArrayInputStream`. you can see my example on how to use it for your purpose. – Jeffrey Blattman Jul 21 '13 at 16:49
  • but your example downloads the entire data. i don't want that. i want to download bytes only when needed . for example: get 1000 bytes (no more than it) , return to the first byte, read all of the bytes. – android developer Jul 21 '13 at 16:51
  • you are in control of how much data you download right? if you only want to read 1000 bytes, abort your read after 1000 bytes. – Jeffrey Blattman Jul 21 '13 at 16:52
  • but then how could i go back and read them again (instead of re-downloading them) and the rest of the bytes? – android developer Jul 21 '13 at 16:53
  • my friend, i gave complete example code of that already. see where i create the `ByteArrayInputStream`? that stream *does* support `reset()`. you can read / reset this as many times as you need. the data is buffered in memory and is not re-read from the source. you create the `ByteArrayInputStream` 1x then read it as many times as you want. – Jeffrey Blattman Jul 21 '13 at 16:56
  • no, as i've already written, you code reads the entire file instead of reading the first bytes that i ask for. only after it has read the entire file, it can handle further requests. – android developer Jul 21 '13 at 16:58
  • no problem. maybe i should update the question to become clearer? what do you think i should write? – android developer Jul 21 '13 at 17:02
  • if you want to read 1k bytes and do something with them, put a counter in your read loop and stop and do whatever you want after 1k bytes. then if you decide you want to keep reading the stream, keep reading. or don't. that being said, there is no method on `BitmapFactory` that lets you pass it the first n bytes and decode meta data about the image. you *have* to give it the entire complete stream so your only option is what i posted. that being said, if you want to read / decode the first N bytes of the compressed image format yourself without `BitmapFactory`, you are welcome to do that. – Jeffrey Blattman Jul 21 '13 at 17:09
  • no, that's not the idea i've talked about. i want to let whoever uses the inputStream to read from it , and later i wish to reset it back to the beginning. the 1k was just an example. also, about bitmapFactory, i don't think it reads the entire file just to get its information, because the information is stored in the header of the file. only the worst case scenario would be to get all of the bytes as you've written. – android developer Jul 21 '13 at 17:53
  • your idea will not work. i can't say this any other way. you either store the bits in memory and make them available for reading and re-reading, or you re-open the stream each time. you cannot reset a network stream unless you wrap it with another stream impl that caches the bytes. you can think what you want about bitmap factory, but the point is, you don't know. if you think you can write a more efficient bitmap factory, by all means, go grab the source and have at it. that is way beyond the scope of what can be covered in a SO question. – Jeffrey Blattman Jul 21 '13 at 18:23
  • now you get it. it's possible, and that's what you've written: "you wrap it with another stream impl that caches the bytes" . question is how i do it, and also how to do it efficiently. there are many streams that wrap other streams (like BufferedInputStream for example) , and it's ok to do such a thing. – android developer Jul 21 '13 at 19:12
  • btw, the BufferedInputStream is supposed to give you the functionality of mark&reset, as written in the API : http://developer.android.com/reference/java/io/BufferedInputStream.html#markSupported() . it supports marking, always. the java API even says it in the description: http://docs.oracle.com/javase/6/docs/api/java/io/BufferedInputStream.html – android developer Jul 21 '13 at 19:37
  • dude, i said it's possible in my first post and wrote the code for you there. you read the stream into a `ByteArrayOutputStream`, get the `byte[]` out of that and construct a `ByteArrayInputStream`, then read / reset that as many times as you like. if you don't need access to the underlying bytes as an array, you can use `BufferedInputStream`. – Jeffrey Blattman Jul 21 '13 at 19:46
  • but how? my code didn't work even though the BufferedInputStream does support the mark&reset... that's why i've written the question. i already know i can read the entire data, but i don't want that since i wish to avoid re-reading data and having to wait for the whole data to be read. – android developer Jul 21 '13 at 19:53
  • now it seems that the code works fine in case it's png, but not when it's jpg. weird. it means my code should work. now the question is why it doesn't work with jpg files. in fact, it seems it works with some jpgs, but can't work with others. here's the test url i've used that doesn't work: http://farm5.staticflickr.com/4031/4637562776_f955d89e3c_o.jpg – android developer Jul 21 '13 at 19:59
  • seems to me that my code doesn't work only on some websites , like the staticflicker.com website, and even then, on some files inside it. i wonder what is the reason for this, and how i can fix it. – android developer Jul 21 '13 at 20:09
-1

Here is a simple method that always works for me :)

 private Bitmap downloadBitmap(String url) {
    // initilize the default HTTP client object
    final DefaultHttpClient client = new DefaultHttpClient();

    //forming a HttoGet request
    final HttpGet getRequest = new HttpGet(url);
    try {

        HttpResponse response = client.execute(getRequest);

        //check 200 OK for success
        final int statusCode = response.getStatusLine().getStatusCode();

        if (statusCode != HttpStatus.SC_OK) {
            Log.w("ImageDownloader", "Error " + statusCode +
                    " while retrieving bitmap from " + url);
            return null;

        }

        final HttpEntity entity = response.getEntity();
        if (entity != null) {
            InputStream inputStream = null;
            try {
                // getting contents from the stream
                inputStream = entity.getContent();

                // decoding stream data back into image Bitmap that android understands
                image = BitmapFactory.decodeStream(inputStream);


            } finally {
                if (inputStream != null) {
                    inputStream.close();
                }
                entity.consumeContent();
            }
        }
    } catch (Exception e) {
        // You Could provide a more explicit error message for IOException
        getRequest.abort();
        Log.e("ImageDownloader", "Something went wrong while" +
                " retrieving bitmap from " + url + e.toString());
    }

    return image;
}
Marcus Gabilheri
  • 1,259
  • 1
  • 14
  • 26