Program crashes on trying to open a file

Question

My android program crashes on this line when the file size is very large. Is there any way I can prevent the program from crashing ?

byte[] myByteArray      =   new byte[(int)mFile.length()];

Additional details :- I am trying to send a file to server. error log-

 E/dalvikvm-heap(29811): Out of memory on a 136309996-byte allocation.

What exactly are you trying to do? It is probably better to read the file with a stream instead of trying to load the whole thing into memory. — brianestey, Jun 17 '13 at 05:48
how big is the file? and how much memory does the phone have? — asgs, Jun 17 '13 at 05:48
you question is too vague. You might need to either post the code or atleast tell what you are intending with the byte array. Bcos the most possible issue is your app runs out of ram memory and gets shutdown. there can be options options of buffered reading, but that again depends on cases. — Naveen Babu, Jun 17 '13 at 05:50
See I need to upload a file to server it could even be 1 GB, the program shouldn't crash. Is there any way out apart from using the NDK? — Sreekanth Karumanaghat, Jun 17 '13 at 05:57
Read the file using a stream and write it out to your network stream immediately. Do not hold anything but the single buffer used in streaming in memory. — brianestey, Jun 17 '13 at 05:59
Uploading a 1GB file to the server? What about the data costs (if applicable) and how much time will it take if the user is on a slower network? Is there any alternate solution? maybe send the final 1000 lines or something instead of sending the whole file. Sounds a bit impractical to me. — midhunhk, Jun 17 '13 at 06:00
@silverback.. 1 GB was just an example, the actual need is to take a photo and upload it to the server , nothing more. The problem is that the photo size could be greater than the RAM of the device. What do you say? — Sreekanth Karumanaghat, Jun 17 '13 at 06:03
@brianestey. A code snippet would be of great help and may be you can add that as answer. :) Thanks — Sreekanth Karumanaghat, Jun 17 '13 at 06:04

score 1 · Accepted Answer · answered Jun 17 '13 at 06:11

You should use a stream when reading the file. Since you've mentioned sending to a server, you should stream that file to the server.

As others have mentioned, you should consider your data size (1GB seems excessive). I haven't tested this, but the basic approach in code would look something like:

// open a stream to the file
FileInputStream fileInputStream = new FileInputStream(filePath);

// open a stream to the server
HttpURLConnection connection = new URL(url).openConnection();
DataOutputStream outputStream = new DataOutputStream(connection.getOutputStream());

byte[] buffer = new byte[BUFFER_SIZE]; // pick some buffer size
int bytesRead = 0;

// continually read from the file into the buffer and immediately write that to output stream
while ((bytesRead = fileInputStream.read(buffer)) != -1) {
    outputStream.write(buffer); 
}

Hope that is clear enough for you to fit to your needs.

Will this example work? http://reecon.wordpress.com/2010/04/25/uploading-files-to-http-server-using-post-android-sdk/ — Sreekanth Karumanaghat, Jun 17 '13 at 06:36
Seems like the same type of code snippet with more of the details filled in. You should try it and see. — brianestey, Jun 17 '13 at 06:41

score 0 · Answer 2 · edited Jun 17 '13 at 05:59

0

In JDK 7 you can use Files.readAllBytes(Path).

Example:

import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;

    Path path = Paths.get("path/to/file");
    byte[] myByteArray = Files.readAllBytes(path);

edited Jun 17 '13 at 05:59

Jibran Khan

3,236
4
37
50

answered Jun 17 '13 at 05:44

Sunil Kumar

7,086
4
32
50

2

How does that differ? I mean any way that much amount of memory has to be allocated. – Sreekanth Karumanaghat Jun 17 '13 at 05:47
1

And can you even use Java 7 features on android? – Axel Jun 17 '13 at 05:53
@Axel I don't think the above code will make a difference, what do you think? – Sreekanth Karumanaghat Jun 17 '13 at 05:56
see the link it is available in java 7 i hope it will work http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#readAllBytes%28java.nio.file.Path%29 – Sunil Kumar Jun 17 '13 at 05:57
1

@SreekanthKarumanaghat Since you still need at least the file size in memory, it will surely OOM as well. The difference I can think of is just that it might not work at all on android even for small files (check the android SKD docs). – Axel Jun 17 '13 at 06:07
@sunil Any way the byte array has to be allocated by the memory in the phone, so this code snippet is not going to work. – Sreekanth Karumanaghat Jun 17 '13 at 06:07
1

@SreekanthKarumanaghat I've added an answer with the basic idea. HTH. – brianestey Jun 17 '13 at 06:11

score 0 · Answer 3 · answered Jun 17 '13 at 05:50

0

Yep. Don't try to read the whole file into memory at once...

If you really need the whole file in memory you might have more luck with allocating dynamic memory for each line and storing the lines in a list. (you might be able to get a bunch of smaller chunks of memory but not one big piece)

Without knowing the context we can't tell, but normally you would parse the file into data structs rather than just storing the whole file in memory.

answered Jun 17 '13 at 05:50

John3136

28,809
4
51
69

See I need to upload a file into server, the program shouldn't crash what ever the size of the file may be. Please provide me with a work around so that I can avoid this crash. – Sreekanth Karumanaghat Jun 17 '13 at 06:01
2

Perhaps you need to upload piece by piece. Without more info we can't help. In theory the file may be bigger then the memory and storage on your device combined, so there is no way to load it all at once then upload. – John3136 Jun 17 '13 at 06:04
All I need to do is to upload a picture taken by a mobile camera to server, what do you say? – Sreekanth Karumanaghat Jun 17 '13 at 06:10

score 0 · Answer 4 · edited May 23 '17 at 12:13

0

Don't try reading the complete file into memory. Instead open a stream and process the file line by line (is it's a text file) or in parts. How that has to be done depends on the problem you are trying to solve.

EDIT: You say you want to upload a file, so please check this question. You don't need to have the complete file in memory.

edited May 23 '17 at 12:13

Community

1
1

answered Jun 17 '13 at 05:51

Axel

13,939
5
50
79

I am trying to upload a file to the server. How can I do that? – Sreekanth Karumanaghat Jun 17 '13 at 05:59
Open a Stream for input, and another for output. Then just copy all the bytes from the input stream to the output stream. – Axel Jun 17 '13 at 06:08

Program crashes on trying to open a file

4 Answers4