Kotlin/anko multiple async tasks

Question

I'm looking for a simple way to start multiple tasks in parallel and wait for all of them to complete.

Consider this c# example:

private static void Main(string[] args)
{
    var urlList = new[] {"http://www.microsoft.com/", "http://www.google.com/", "http://www.apple.com/" };
    var result = GetHtml(urlList);
}

private static List<string> GetHtml(string[] urlList)
{
    var tasks = new List<Task>();
    var output = new List<string>();

    foreach (var url in urlList)
    {
        var task = new Task(() =>
        {
            var html = new WebClient().DownloadString(url);
            output.Add(html);
        });

        tasks.Add(task);

        //starts task in a separate thread (doesn't block anything)
        task.Start();
    }

    //waits for all tasks (running in parallel) to complete before exiting method
    Task.WaitAll(tasks.ToArray());

    return output;
}

GetHtml method downloads multiple web-pages in parallel and returns a list of html strings.

How can I achieve this using kotlin/anko?

private fun GetHtml(urlList: Array<String>): ArrayList<String> {

    val tasks = ArrayList<Future<Unit>>()
    val output = ArrayList<String>()

    for (url in urlList) {
        val task = async() {
            //some java-code that downloads html from <url>, doesn't matter for now
            output.add("html for $url")
        }
        tasks.add(task)
    }

    //this is NOT parallel execution
    for (task in tasks) {
        task.get()           
    }

    //tasks.getall() ?? 

    return output
}

Perhaps there's a better way, but you can [specify an `ExecutorService` for `async`](https://github.com/Kotlin/anko/blob/master/doc/ADVANCED.md#asynchronous-tasks), and `ExecutorService` has an [`awaitTermination` method](https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#awaitTermination(long,%20java.util.concurrent.TimeUnit)). — Michael, Jun 01 '16 at 14:59
With Kotlin 1.1.1, you can `await` one or more jobs, see http://stackoverflow.com/a/43151714/882912 — KTCO, Apr 19 '17 at 02:00

Fire095 · Accepted Answer · 2016-06-02T12:24:31.983

Based on input from Michael and pdegand59, here's a working solution:

private fun GetHtml(urlList: Array<String>): ArrayList<String> {

    val pool = Executors.newFixedThreadPool(urlList.count())
    val countDownLatch = CountDownLatch(urlList.count())

    val output = ArrayList<String>()

    for (url in urlList) {  

        async(pool, {
            //some java-code that downloads html for <url>
            output.add("html for $url")
            countDownLatch.countDown()
        })      
    }

    countDownLatch.await()

    return output
}

pdegand59 · Answer 2 · 2016-06-02T11:39:58.527

1

I would suggest to use CountDownLatch in a separate thread to avoid blocking the main thread during all the parallel downloads :

private fun downloadAllHtml(urlList: Array<String>) {
  val output = ArrayList<String>()
  val countDownLatch = CountDownLatch(urlList.length)

  async(Executors.newSingleThreadExecutor() {
    countDownLatch.await()
    uiThread {
      onAllHtmlDownloaded(output)
    }
  }

  urlList.forEach { url: String ->
    async() {
      // download stuff
      output.add("stuff downloaded")
      countDownLatch.countDown()
    }
  }
  // this method ends instantly, not blocking the main thread
}

private fun onAllHtmlDownloaded(output: ArrayList<String>) {
  // all your html, on the main thread
}

You may have to add some try/catch. The IDE will help you ;)

edited Jun 02 '16 at 11:39

answered Jun 02 '16 at 08:31

pdegand59

12,776
8
51
58

CountDownLatch looks great, but the code you provided does not work, unfortunately. onAllHtmlDownloaded method is never reached because countDownLatch.await() causes the process to wait indefinitely. What I did was create a separate ThreadPool and use countDownLatch.await(), this worked fine. – Fire095 Jun 02 '16 at 11:27
I updated the code with moving the async call with await before the loop and running this async on another Executor. – pdegand59 Jun 02 '16 at 11:39
Thanks for your input! The problem here is not in the CountDownLatch but rather in async() blocks which download html. Blocking the main thread is not a concern in this scenario, so lets keep it as simple as possible:) I've updated my answer and gave you credit – Fire095 Jun 02 '16 at 12:23

Kotlin/anko multiple async tasks

2 Answers2