how to improve performance when tons of AsyncTasks running

Question

I have been writing a file download module, when user select some files and click download button, it will create as many AsyncTasks as the files. All these files were added to an array. Then I traverse the array, create an AsyncTask for the file, call execute method of each AsyncTask instance.

I tested this module by downloading 20 photos, 1 MB each, turned out it can successfully download these files and being stable. But problem comes when I add thousands of files, for example 2000 photos, to download.

When user click to download those 2000 photos, it will create 2000 AsyncTasks and call execute method of each immediately. That`s not the best practice, or even cause OOM, I think. Because it starts 2000 threads at the same time, though only the top 5 AsyncTask instances will invoke their doInbackground method on Kitkat according to official document, 2000 threads were created and allocated resources already (right?) which may leads up to OOM.

For a better performance, I plan to manage these AsyncTasks in a queue, only poll top five instances and execute them, anyone is done executing, it sends a notification and poll a substitute one out until all remaining AsyncTasks have been executed. I wonder if it is right to think like this. Thanks in advance!

method for add download task

/**
 * Add a new download task
 */
public int addDownloadTask(Account account,
                           String repoName,
                           String repoID,
                           String path) {
    // omit lines...
    DownloadTask task = new DownloadTask(account, repoName, repoID, path);
    // execute download task serially
    task.execute();
    return task.getTaskID();
}

call addDownloadTask in a loop

for (SeafDirent seafDirent : dirents) {
            if (!seafDirent.isDir()) {
                File localCachedFile = dataManager.getLocalCachedFile(repoName, repoID, seafDirent.name), seafDirent.id);
                if (localCachedFile == null) {
                    txService.addDownloadTask(account, repoName, repoID, Utils.pathJoin(filePath, seafDirent.name));
                }
            }
        }

Good way to think like this. If you don't mind, can you put some code here, so that i can get some idea. — Android Killer, Dec 01 '14 at 06:02
Why not, instead of firing 2000 AsyncTasks, fire one AsyncTask that traverses each download? — Tim Malseed, Dec 01 '14 at 06:02
Did you think about a thread pool which can manage the queue in the way you want? Or do you need the UIRunning methods of the asynctask? That way you doesn't have to recreate an instance of a new asynctask 1999times. Would have impact on performance.. — Mike, Dec 01 '14 at 06:03
@Mike I need to publish downloading progress respectively, I used taskID as an identifier as you can see from the code above. — Logan Guo, Dec 01 '14 at 06:33
That's no problem you can notify the UI or main Thread after one download has been finished.. If i understand you correctly. No need of using asynctask. You are creating a lot of instances .. Which can cause OOM or heavy performance impacts. With a thread pool you reuse existing threads.. Which is a better way due to performance and you can setup many other things about how to handle thread queue. No need to reinvent the wheel ;) — Mike, Dec 01 '14 at 06:40
@tmalseed because I need to update downloading progress instantly — Logan Guo, Dec 01 '14 at 06:49
Out of memory @android killer it depends on how he is initiating the 2000 asynctask. All at once or a few time over time. — Mike, Dec 01 '14 at 06:54
@LoganGuo I'm just throwing up suggestions here, but can't you do that in onProgressUpdated() in your AsyncTask? — Tim Malseed, Dec 01 '14 at 07:06

score 1 · Accepted Answer · answered Dec 01 '14 at 06:36

1

You should create your own Queue and its Executor according to your requirements to manage Threads.

Please have a look at example of ThreadPool creation to manage multiple threads , ThreadPool , ThreadPool Executor

answered Dec 01 '14 at 06:36

Mehul Joisar

15,348
6
48
57

Thanks for your comment, this example is about managing runnable types but AsyncTask, so I wonder if it is possible to rewrite it – Logan Guo Dec 01 '14 at 06:51
@LoganGuoz: `AsyncTask` is nothing but a thread, and so is the `Runnable`. Android framework uses `ThreadPoolExecutor` to manage predefined number of threads, you may specify your limits if you make your own `ThreadPoolExecutor` as mentioned in examples. – Mehul Joisar Dec 01 '14 at 07:00
I confused about how to convert from Asynctask to Runnable type – Logan Guo Dec 01 '14 at 07:06
@LoganGuo: whatever you write in `doInBackground` of `AsyncTask` should be written in `run` method of `Runnable` – Mehul Joisar Dec 01 '14 at 07:10
@LoganGuo: just download [this example](https://developer.android.com/shareables/training/ThreadSample.zip) and have a look at code of `PhotoDownloadRunnable.java` – Mehul Joisar Dec 01 '14 at 07:13
if I change to use Runnable, how could I publish downloads progress? – Logan Guo Dec 01 '14 at 07:13

score 0 · Answer 2 · edited May 23 '17 at 12:20

Async tasks use a thread pool executor pattern. You cannot have more than 128 concurrently running tasks.

This thread explains it clearly: Running multiple AsyncTasks at the same time -- not possible?

However, your question is about creating the async task instances (which are essentially objects). When you create objects in loop, there would certainly be a point when the system runs out of memory.

I also think that OOM is more likely to occur because of the images of that size getting loaded into memory rather than object creation. You may want to use weak references (http://android-developers.blogspot.in/2010/07/multithreading-for-performance.html)

score 0 · Answer 3 · answered Dec 01 '14 at 06:29

0

try bolts framework, it is from facebook and parse, the main purpose is to help you manage your tasks.

https://github.com/BoltsFramework/Bolts-Android

answered Dec 01 '14 at 06:29

Rene Xu

1,093
1
9
14

how to improve performance when tons of AsyncTasks running

3 Answers3