How to make for-in loop wait for data fetch function to complete

Question

I am trying to fetch bunch of data with for in loop function, but it doesn't return data in correct orders. It looks like some data take longer to fetch and so they are mixed up in an array where I need to have all the data in correct order. So, I used DispatchGroup. However, it's not working. Can you please let me know what I am doing wrong here? Spent past 10 hours searching for a solution... below is my code.

@IBAction func parseXMLTapped(_ sender: Any) {

    let codeArray = codes[0]
    for code in codeArray {
                    
              self.fetchData(code)
                    
     }
                
      dispatchGroup.notify(queue: .main) {
          print(self.dataToAddArray)
          print("Complete.")
     }

}

private func fetchData(_ code: String) {
        dispatchGroup.enter()
        print("count: \(count)")

        let dataParser = DataParser()
        dataParser.parseData(url: url) { (dataItems) in
            self.dataItems = dataItems
            
            print("Index #\(self.count): \(self.dataItems)")
            self.dataToAddArray.append(self.dataItems)

        }
        self.dispatchGroup.leave()
        
        dispatchGroup.enter()
        self.count += 1
        dispatchGroup.leave()
        
    }

Dispath group can't help you in this case. You should using DispathSemaphore for this. — goat_herd, Aug 20 '20 at 07:47
With respect, using semaphore to make the tasks run sequentially, while it works, is almost always the wrong answer. If the problem is that they are finishing in the wrong order, then order the results rather than forcing them to run sequentially. Don't sacrifice all the performance benefits of concurrency! — Rob, Jun 14 '21 at 18:03

score 4 · Answer 1 · answered Aug 20 '20 at 07:40

The problem with asynchronous functions is that you can never know in which order the blocks return. If you need to preserve the order, use indices like so:

let dispatchGroup = DispatchGroup()
var dataToAddArray = [String](repeating: "", count: codeArray.count)
for (index, code) in codeArray.enumerated() {
    dispatchGroup.enter()
    DataParser().parseData(url: url) { dataItems in
        dataToAddArray[index] = dataItems
        dispatchGroup.leave()
    }
}
dispatchGroup.notify(queue: .main) {
    print("Complete"
}

Also in your example you are calling dispatchGroup.leave() before the asynchronous block has even finished. That would also yield wrong results.

Rob · Answer 2 · 2021-06-17T17:01:08.980

Using semaphores to eliminate all concurrency solves the order issue, but with a large performance penalty. Dennis has the right idea, namely, rather than sacrificing concurrency, instead, just sort the results.

That having been said, I would probably use a dictionary:

let group = DispatchGroup()
var results: [String: [DataItem]]                        // you didn't say what `dataItems` was, so I'll assume it's an array of `DataItem` objects; but this detail isn't material to the broader question

for code in codes {
    group.enter()
    DataParser().parseData(url: url) { dataItems in
        results[code] = dataItems                        // if parseData doesn't already uses the main queue for its completion handler, then dispatch these two lines to the main queue
        group.leave()
    }
}

group.notify(queue: .main) {
    let sortedResults = codes.compactMap { results[$0] } // this very efficiently gets the results in the right order
    // do something with sortedResults
}

Now, I might advise constraining the degree of concurrency (e.g. maybe you want to constrain this to the number of CPUs or some reasonable fixed number (e.g. 4 or 6). That is a separate question. But I would advise against sacrificing concurrency just to get the results in the right order.

score 1 · Accepted Answer · edited Jun 14 '21 at 17:36

1

In this case, using DispatchSemaphore:

let semaphore = DispatchSemaphore(value: 0)

DispatchQueue.global().async {
    for code in codeArray {
        self.fetchData(code)
        semaphore.wait()
    }
}

private func fetchData(_ code: String) {
    print("count: \(count)")

    let dataParser = DataParser()
    dataParser.parseData(url: url) { (dataItems) in
        self.dataItems = dataItems

        print("Index #\(self.count): \(self.dataItems)")
        self.dataToAddArray.append(self.dataItems)
        semaphore.signal()
    }
}

edited Jun 14 '21 at 17:36

Rob

415,655
72
787
1,044

answered Aug 20 '20 at 07:59

goat_herd

571
3
13

Thank you. I spent past few days to study semaphore and learned it is very useful. Your answer helped me a lot. Very much appreciated. – Gilbert Tyler Aug 23 '20 at 21:28

How to make for-in loop wait for data fetch function to complete

3 Answers3