Firebase Storage RejectedExecutionException exception

Question

In my project I am using Firebase Storage to download some image files - 154 images to be exact. For some unknown reason to me I only manage to download only around 130 files and then I get an exception stating:

java.util.concurrent.RejectedExecutionException: Task com.google.firebase.storage.zzs@15c1d59d rejected from java.util.concurrent.ThreadPoolExecutor@270bfa12[Running, pool size = 3, active threads = 3, queued tasks = 128, completed tasks = 5]
    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2011)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:793)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1339)
    at com.google.firebase.storage.zzu.zzv(Unknown Source)
    at com.google.firebase.storage.FileDownloadTask.schedule(Unknown Source)
    at com.google.firebase.storage.StorageTask.zzcny(Unknown Source)
    at com.google.firebase.storage.StorageReference.getFile(Unknown Source)
    at com.google.firebase.storage.StorageReference.getFile(Unknown Source)

I am using for loop to call StorageReference.getFile() and download the file in specific location like this:

        for (int i = 0; i < filesToDownload.size(); i++) {
            String image = filesToDownload.get(i);

            final File localFile = new 
               File(DeviceStorage.getExternalStorageDir(dirPath)
               + File.separator + fileName);
            StorageReference downloadPhotoRef = 
               getReference().child(folderName + File.separator + fileName);

            downloadPhotoRef.getFile(localFile);
            downloadFile(context, image);
        }

I am really puzzled why does this happen. I couldn't find any solutions and other people having the same problem. I run this code in a Service. The only way I don't get the error is if I'm downloading less than 130 files, but this is no good to me.

Please help.

Gazihan Alankus · Accepted Answer · 2019-03-19T04:52:40.677

You are starting too many parallel download tasks. The threading system gives up as a result. More about that here: https://stackoverflow.com/a/8183463/679553

You should try to serialize parts of this process, maybe download them 10 by 10? .continueWith() here is a good start: https://firebase.googleblog.com/2016/09/become-a-firebase-taskmaster-part-3_29.html

At the moment your getFile() call spawns a new download task and returns without blocking. Instead of starting the next download with the next iteration of the for loop, you can start it after the previous download is done using .continueWith().

One way to implement this is to create a function that returns a task and calling itself back in the last continueWith block. Kind of like the C# example form here: https://stackoverflow.com/a/49043177/679553

I modified an example from that link to make it closer to your case. I'm on mobile so this is the best I can do at the moment:) You can make the ref be a field in your class and update it inside the continueWith blocks.

Task DoStuff()
{
    return ref.getFile(() => DoWork1())
        .ContinueWith((t1) => DoWork2())
        .ContinueWith(t2 => DoWork3())
        .ContinueWith(t3=> DoStuff());
}

Ok this is great start. The only thing I still to get is how to implement .continueWith() in my case. All the examples in the firebase blog article are with known number of Tasks, unlike my case where I have array whom size is dynamic value. I tried another thing: initializing the "i" variable in the "addOnSuccessListener" of the getFile() but for some reason the execution never goes into addOnSuccessListener — Binev, Mar 18 '19 at 22:07
I edited my answer starting with "One way...". I'm on mobile right now but I did this in the past and something like this will work. — Gazihan Alankus, Mar 19 '19 at 04:54
Thanks I think this solves my issue! You were very helpful indeed! — Binev, Mar 19 '19 at 14:36

Firebase Storage RejectedExecutionException exception

1 Answers1