Finish all asynchronous requests before loading data?

Question

I have run into an issue where I have multiple asynchronous requests occuring which grab images and information from the Facebook API and my Firebase database. I want to perform all my asynchronous requests, then store all that data that I grabbed from the Facebook API/Firebase database into one entire object which I can quickly load. I have set up completion handlers for every asynchronous request which I thought forces the program to "wait" until the request is complete and then have the program continue, but that doesn't seem to work for me. Below is my attempt:

func setupEvents(completion: (result: Bool, Event: Event) -> Void){
    // Get a reference to Events
    eventsReference = Firebase(url:"<DB Name>")
    eventAttendeesRef = Firebase(url:"<DB Name>")

    //Read the data at our posts reference
    println("Event References: \(eventsReference)")
    eventsReference.observeEventType(FEventType.ChildAdded, withBlock: { (snapshot) -> Void in

        let eventName = snapshot.value["eventName"] as? String
        let eventLocation = snapshot.value["eventLocation"] as? String
        let eventCreator = snapshot.value["eventCreator"] as? String

        var attendees: NSMutableDictionary = [:]
        var attendeesImages = [UIImage]()
        let attendee: NSMutableDictionary = [:]

        let group = dispatch_group_create()

        //Get attendees first
        dispatch_group_enter(group)
        self.getAttendees(snapshot.key as String, completion:{ (result, name, objectID) -> Void in
            if(result == true){
                println("Finished grabbing \(name!) \(objectID!)")
                attendees.addEntriesFromDictionary(attendee as [NSObject : AnyObject])
            }
            else {
                println("False")
            }
            dispatch_group_leave(group)
        })

        //Get attendees photos
        dispatch_group_enter(group)
        self.getAttendeesPictures(attendee, completion: { (result, image) -> Void in
            if result == true {
                println("Finished getting attendee photos. Now to store into Event object.")
                attendeesImages.append(image!)
            }
            else{
                println("false")
            }
            dispatch_group_leave(group)
        })

        dispatch_group_notify(group, dispatch_get_main_queue()) {
            println("both requests done")
            //Maintain array snapshot keys
            self.eventIDs.append(snapshot.key)

            if snapshot != nil {
                let event = Event(eventName: eventName, eventLocation:eventLocation, eventPhoto:eventPhoto, fromDate:fromDate, fromTime:fromTime, toDate:toDate, toTime:toTime, attendees: attendees, attendeesImages:attendeesImages, attendeesImagesTest: attendeesImagesTest, privacy:privacy, eventCreator: eventCreator, eventCreatorID: eventCreatorID)
                println("Event: \(event)")
                completion(result: true, Event: event)
            }
        }

        }) { (error) -> Void in
            println(error.description)
    }
}

I know I have my completion handlers set correctly as I have tested in my program. However, what I want is that only after both the getAttendees and getAttendeesPictures function completes, I then want to store all the information I grabbed the snapshot, getAttendees, and getAttendeesPictures function and store them into an event object. Any ideas on how to accomplish this? I've tried to look into dispatch_groups to help me handle this via this link: Checking for multiple asynchronous responses from Alamofire and Swift but my program seems to only execute the getAttendees function but not the getAttendeesPictures function. Below are also the getAttendees and getAttendeesPictures functions:

func getAttendees(child: String, completion: (result: Bool, name: String?, objectID: String?) -> Void){
    //Get event attendees of particular event
    var attendeesReference = self.eventAttendeesRef.childByAppendingPath(child)
    println("Loading event attendees")
    //Get all event attendees
    attendeesReference.observeEventType(FEventType.ChildAdded, withBlock: { (snapshot) -> Void in
        let name = snapshot.value.objectForKey("name") as? String
        let objectID = snapshot.value.objectForKey("objectID") as? String
        println("Name: \(name) Object ID: \(objectID)")
        completion(result: true, name: name, objectID: objectID)
        }) { (error) -> Void in
            println(error.description)
    }

 func getAttendeesPictures(attendees: NSMutableDictionary, completion: (result: Bool, image: UIImage?)-> Void){
    println("Attendees Count: \(attendees.count)")
    for (key, value) in attendees{
        let url = NSURL(string: "https://graph.facebook.com/\(key)/picture?type=large")
        println("URL: \(url)")
        let urlRequest = NSURLRequest(URL: url!)
        //Asynchronous request to display image
        NSURLConnection.sendAsynchronousRequest(urlRequest, queue: NSOperationQueue.mainQueue()) { (response:NSURLResponse!, data:NSData!, error:NSError!) -> Void in
            if error != nil{
                println("Error: \(error)")
            }
            // Display the image
            let image = UIImage(data: data)
            if(image != nil){
                completion(result: true, image: image)
            }
        }
    }
}

Answer 1

For users seeking answer to question in title then use of dispatch_group and GCD outlined here: i.e embedding one group inside the notification method of another dispatch_group is valid. Another way to go at a higher level would be NSOperations and dependencies which would also give further control such as canceling operations.

Outline:

func doStuffonObjectsProcessAndComplete(arrayOfObjectsToProcess: Array) -> Void){

    let firstGroup = dispatch_group_create()

    for object in arrayOfObjectsToProcess {

        dispatch_group_enter(firstGroup)

        doStuffToObject(object, completion:{ (success) in
            if(success){
                // doing stuff success
            }
            else {
                // doing stuff fail
            }
            // regardless, we leave the group letting GCD know we finished this bit of work
            dispatch_group_leave(firstGroup)
        })
    }

    // called once all code blocks entered into group have left
    dispatch_group_notify(firstGroup, dispatch_get_main_queue()) {

        let processGroup = dispatch_group_create()

        for object in arrayOfObjectsToProcess {

            dispatch_group_enter(processGroup)

            processObject(object, completion:{ (success) in
                if(success){
                    // processing stuff success
                }
                else {
                    // processing stuff fail
                }
                // regardless, we leave the group letting GCD know we finished this bit of work
                dispatch_group_leave(processGroup)
            })
        }

        dispatch_group_notify(processGroup, dispatch_get_main_queue()) {
            print("All Done and Processed, so load data now")
        }
    }
}

The remainder of this answer is specific to this codebase.

There seem to be a few problems here: The getAttendees function takes an event child and returns an objectID and Name which are both Strings? Shouldn't this method return an array of attendees? If not, then what is the objectID that is returned?

Once an array of attendees is returned, then you can process them in a group to get the pictures.

The getAttendeesPictures eventually returns UIImages from Facebook. It's probably best to cache these out to the disk and pass path ref - keeping all these fetched images around is bad for memory, and depending on size and number, may quickly lead to problems.

Some examples:

func getAttendees(child: String, completion: (result: Bool, attendees: Array?) -> Void){

    let newArrayOfAttendees = []()

    // Get event attendees of particular event

    // process attendees and package into an Array (or Dictionary)

    // completion
    completion(true, attendees: newArrayOfAttendees)
}

func getAttendeesPictures(attendees: Array, completion: (result: Bool, attendees: Array)-> Void){

    println("Attendees Count: \(attendees.count)")

    let picturesGroup = dispatch_group_create()

    for attendee in attendees{

       // for each attendee enter group
       dispatch_group_enter(picturesGroup)

       let key = attendee.objectID

       let url = NSURL(string: "https://graph.facebook.com/\(key)/picture?type=large")

        let urlRequest = NSURLRequest(URL: url!)

        //Asynchronous request to display image
        NSURLConnection.sendAsynchronousRequest(urlRequest, queue: NSOperationQueue.mainQueue()) { (response:NSURLResponse!, data:NSData!, error:NSError!) -> Void in
            if error != nil{
                println("Error: \(error)")
            }

            // Display the image
            let image = UIImage(data: data)
            if(image != nil){
               attendee.image = image
            }

            dispatch_group_leave(picturesGroup)
        }
    }

    dispatch_group_notify(picturesGroup, dispatch_get_main_queue()) {
         completion(true, attendees: attendees)
    }
}

func setupEvents(completion: (result: Bool, Event: Event) -> Void){

    // get event info and then for each event...

    getAttendees(child:snapshot.key, completion: { (result, attendeesReturned) in
        if result {
            self.getAttendeesPictures(attendees: attendeesReturned,         completion: { (result, attendees) in

              // do something with completed array and attendees


            }
        }
        else {

        }
    })

}

The above code is just an outline, but hopefully points you in the right direction.

Answer 2

The two requests are executing at the same time, so there is no attendees to get pictures from when the second request executes, if the getAttendees completion closure is going to be called multiple times then you can do something like this:

let group = dispatch_group_create()

for key in keys {
   dispatch_group_enter(group)
   self.getAttendee(key as String, completion:{ (result, attendee) in
      if(result == true){
         attendees.addEntriesFromDictionary(attendee)
         self.getAttendeesPictures(attendee, completion: { (result, image) in
           if result == true {
              attendeesImages.append(image!)
           }
           dispatch_group_leave(group)
         })
      } else {
         dispatch_group_leave(group)
      }            
   })
}

dispatch_group_notify(group, dispatch_get_main_queue()) {}

If the result of the first request is the complete set of attendees you don't even need to use GCD, just call getAttendeesPictures inside the completion closure.

This code doesn't exactly uses the same variables and methods of the original code, it only gives the idea.

Hope it helps!

Answer 3

While there is definitely solution with using GCD and stuff around it, synchronization in general is pain and the more your code gets complicated, the more problems it will start showing - but I think there is one-for-all solution to that: Bolts framework from Facebook (both for android na iOS)

Bolts Framework usage

So what is so magical about it? Well, it lets you create "Tasks", and then chain them. The method in particular that you are interested in is taskForCompletionOfAllTasks: , which is made for parallel processing, just what you need. I wrote a little example for you which you can adjust to your needs:

func fetchAllInformation() -> BFTask {

    // First, create all tasks (if you need more, than just create more, it is as easy as that
    var task1 = BFTaskCompletionSource()
    var task2 = BFTaskCompletionSource()
    var tasks = [task1, task2]

    // What you do, is you set result / error to tasks and the propagate in the chain upwards (it is either result, or error)
    // You run task 1 in background
    API.instance.fetchFirstDetailsInBackgroundWithBlock {
        (object: AnyObject!, error: NSError!) -> Void in

        // On error or on success, you assign result to task (whatever you want)
        if error == nil {
            task1.setResult(object)
        } else {
            task1.setError(error)
        }
    }

    // You run task 2 in background
    API.instance.fetchSecondDetailsInBackgroundWithBlock {
        (object: AnyObject!, error: NSError!) -> Void in

        // On error or on success, you assign result to task (whatever you want)
        if error == nil {
            task2.setResult(object)
        } else {
            task2.setError(error)
        }
    }

    // Now you return new task, which will continue ONLY if all the tasks ended
    return BFTask(forCompletionOfAllTasks: tasks)
}

Once you have main method done, you can use bolts chaining magic:

func processFullObject() {

    // Once you have main method done, you can use bolts chaining magic
    self.fetchAllInformation().continueWithBlock { (task : BFTask!) -> AnyObject! in

        // All the information fetched, do something with result and probably with information along the way
        self.updateObject()
    }
}

The Bolts framework documentation / README covers basically everything there is to know about it and it is quite extensive, so I would suggest you to go through it - it is very easy to use once you get the basics. I personally use it for exactly this, and it is a blast. This answer will hopefully provide you with different solution and approach, possibly a cleaner one.

Answer 4

There is something wrong with this conceptually. It sounds like you want to wait until both of these functions complete before doing something else, but what you haven't explained is that getAttendeesPictures depends on the outcome of getAttendees. That means what you really want to do it execute one asynchronous block, then execute a second asynchronous block with the output of the first, and then execute your final completion block when both are finished.

GCD is not particularly suited for this; you're better of using NSOperationQueue with NSBlockOperations. There are two distinct advantages to this over GCD:

1. NSOperation uses familiar object-oriented syntax compared to GCD's c-type functions, so it's pretty easy to write and understand.
2. Operations in the queue can have explicit dependencies on one another, so you can make it clear that e.g. operation B will only be executed after operation A is complete.

There is a great writeup of this by NSHipster which I'd recommend you go read. It's talked about mostly in the abstract, but what you want to do is use NSBlockOperation to create two block operations, one for executing getAttendees and one for executing getAttendeesPictures, and then make it explicit that the second block depends on the first before adding them both to a queue. They will then both execute and you can use a completion block on the second operation to do something once both have completed.

Dave Roberts is right in his response though: an immediate problem with the code is that you don't use the output of the getAttendees function to actually create any attendees. Perhaps this part of the code is missing, but from what I can see the name and objectID are just printed out. If you want to pass something useful into the getAttendeesPictures function you will need to fix this part first.

Answer 5

This is off the top of my head. The idea is to read and handle new asyc data only when all of the nested blocks complete.

We leverage a while loop to handle waiting for a signal to read the next set of data.

The outside while loop continues as long as done equals false. And nothing is really going on, other than consuming cpu cycles while it waits. The if inside the loop will only be trigged (set to true) when all of the attendees have been read.

Meanwhile inside the loop we work through nested blocks, reading in the attendee and then when that completes, read their picture, and when that completes read the firebase data. Finally once we have all data from the prior blocks we stuff the data into an object which is then added to the dictionary. At that time it is determined if we are finished reading attendees and if so, bail completely. If not, we read the next attendee.

(this is conceptual)

done = false
readyToReadNextAttendee = true
      while ( done == false )
      {
        if (readyToReadNextAttendee == true ) {
          readyToReadNextAttendee = false
          readAttendee
           readPicture
            readFirebase {
              putDataIntoObject
              addObjectToDictionary
              if finishedReadingAttendees {
                 done = true
              } else {
                 readyToReadNextAttendee = true
              }
            }
        }
      }

If you have the option of reading in all of the attendees first, you could iterate over and array as well, not reading the next index until readyToReadNextAttendee = true

Answer 6

One Idea i have used is to place an if statement check inside the query statement call back and place the query statement call back in a for loop (so you can loop through all of your queries), so the if statement should check if this the last call back expected, then you should execute a return statement or a deferred.resolve statement, the following is a concept code.

var list=fooKeys //list of keys (requests) i want to fetch form firebase
var array=[]  // This is the array that will hold the result of all requests 
for(i=xyz;loop breaking condition; i++){
    Ref = new Firebase("https://yourlink.firebaseio.com/foo/" + fooKeys[i]);
   Ref.once("value", function (data) {
       array.push(data.val());
       if(loop breaking condition == true){
           //This mean that we looped over all items
           return array;  //or deferred.resolve(array);
       }
   })
}

Putting this code in a function and call it asynchronously will give you the ability to wait for the whole results before proceed in doing other stuff.

Hope you (and the others) find this beneficial.