AS3 fileStream appears to read the file into memory

Question

I am writing a process where users will need to select a file that far exceeds their availble RAM and have that file broken up into small chunks (for upload).

I'm able to create a File reference to said file, but when I try to pass it to the fileStream, it appears to try to read the thing into memory before acting on it.

Is there a way to get fileStream to just take the reference to the file and then utilize readBytes the way it's documented?

Here is my code... it's called when the user selects the File in the browser dialogue.

private function selectHandler(event:Event):void {
        var file:File = File(event.target);
        trace("selectHandler: name=" + file.name );

        var stream:FileStream = new FileStream();

        var f:File = event.target as File;
        stream.open(f, FileMode.READ);  //here the process will lock up if the file you pass it is too large.
        var bytes:ByteArray = new ByteArray();
        stream.readBytes(bytes,0,1024);
        trace(bytes);
        stream.close();
}

Much obliged, in advance.

Yevgeny Simkin · Accepted Answer · 2011-02-07T11:14:43.950

So, the solution...

 stream.readAhead = 10000;// some reasonable number
 stream.openAsync(f, FileMode.READ);  //here the process will no longer lock up, if the above chunk is set to a number that Flash can handle.



   //then in your PROGRESS listener you read the bytes into a byteArray
  if(target.availableBytes >= 10000){//you need this because progress gets called many times before the full chunk is read. You only want to use it when you have the full chunk. (you also will want to keep track of the total read and when that total + chunk > fileSize, you'll want to adjust your chunk to read in that last bunch of bytes.
    var bytes:ByteArray = new ByteArray();
    stream.readBytes(bytes,0,event.target.availableBytes); //this will provoke the next segment to get read as well
  }

score 1 · Answer 2 · answered Jan 24 '11 at 22:44

1

There's an openAsync method in the FileStream class, which, of course, asynchronously sends you parts of the file, but I don't know if it will be useful to you, since you will still have to store the parts in memory for later upload. The alternative I think would be to use NativeProcess to retrieve determined parts of the file, through for example, Java.

I had a similar situation where I needed to retrieve the hash of the first and last 64 bytes of some fairly big files (up to 10gb) and the latter technique (NativeProcess>Java) worked like a charm. I'm a complete n00b in Java, but if you want I can post said code (I copied/pasted snippets until it worked, so it's most likely quite rough).

answered Jan 24 '11 at 22:44

Cay

3,804
2
20
27

Thanks... no, you're right, in Java this is a no-brainer since the source file is not read into memory. the async variation of the open method appears to give you access to the bytes read, but wants to read the file into memory anyway (it just lets you play with what it read already). This is not useful for a file that exceeds the RAM on a given machine. – Yevgeny Simkin Jan 24 '11 at 23:17
So, openAsync is one part of the solution, but it turns out that there's another critical part. The readAhead property. When left unset it defaults to infinity and will attempt to read the entire file into memory. However, if you set it to some reasonable amount (10 megs works nicely) it will then read just 10 megs at a time. – Yevgeny Simkin Jan 31 '11 at 06:41
you're welcome. Please add this info to your answer in case someone else comes looking, and I will give you the green checkmark. – Yevgeny Simkin Feb 04 '11 at 22:19
Don't worry, you can just accept your own answer, which is more correct ;) – Cay Feb 04 '11 at 22:31

AS3 fileStream appears to read the file into memory

2 Answers2

Linked