3

I have some code which requires the use of blocks. The block fetches a number of data items from a web service, and then possibly needs to fetch more, and then more again after that, then returns all of the data items once it has all required. I'm unsure how to put this into code. Here's an example of what I mean:

NSMutableArray *array = [[NSMutableArray alloc] init];

[webService getLatestItemsWithCount:50 completion:^(NSArray *objects) {
    //Some code to deal with these items.

    if (moreItemsNeeded == YES) {
        //I now need it to loop this block until I'm done
    }
}];

How can I get this to work?

EDIT:

Ok, this is what i'm working with - it's the Evernote API. It should be a better example of what I need:

[noteStore findNotesMetadataWithFilter:filter
                                offset:0
                              maxNotes:100
                            resultSpec:resultSpec
                               success:^(EDAMNotesMetadataList *metadataList) {
    for (EDAMNoteMetadata *metadata in metadataList.notes) {
        NSDate *timestamp = [NSDate endateFromEDAMTimestamp:metadata.updated];

        if (timestamp.timeIntervalSince1970 > date.timeIntervalSince1970) {
            [array addObject:metadata];
        }
        else {
            arrayComplete = YES;
        }
    }

    //I need it to loop this code, increasing the offset, until the array is complete.

}failure:^(NSError *error) {
    NSLog(@"Failure: %@", error);
}];
Ramy Al Zuhouri
  • 21,580
  • 26
  • 105
  • 187
Andrew
  • 15,935
  • 28
  • 121
  • 203
  • 1
    How do you know if you need to fetch more items? What do you do with the items? How is this process started in the first place? On what thread or queue do you run it? What API do you use to fetch the items from the server? – rob mayoff Jul 01 '13 at 21:57
  • It checks the date on each item returned. If it's earlier than a certain date, it needs to collect more items before it returns. – Andrew Jul 01 '13 at 22:03

4 Answers4

4

You should create a variable that references the block to make possible the recursive invocation. It must be noted that at the moment that you assign the block, it is still nil, so if you call it inside the block itself (aka recursively), you'll get a crash while trying to execute a nil block. So the block should have a *__block* storage:

void (^__block myBlock) (NSArray*) = ^(NSArray *objects) {
    //Some code to deal with these items.

    if (moreItemsNeeded == YES) {
        //I now need it to loop this block until I'm done
        myBlock(objects);
        myBlock= nil; // Avoid retain cycle
    }
}];
[webService getLatestItemsWithCount:50 completion: myBlock];

The block in your specific case is "translated" as this one:

void (^__block handler) (EDAMNotesMetadataList)= ^(EDAMNotesMetadataList* metadataList) {
    for (EDAMNoteMetadata *metadata in metadataList.notes) {
        NSDate *timestamp = [NSDate endateFromEDAMTimestamp:metadata.updated];

        if (timestamp.timeIntervalSince1970 > date.timeIntervalSince1970) {
            [array addObject:metadata];
        }
        else {
            arrayComplete = YES;
        }
    }

    //I need it to loop this code, increasing the offset, until the array is complete.
    if(!arrayComplete)
        handler(metadataList);
    handler= nil; // Avoid retain cycle
};

Then you can normally call that method passing myBlock as argument.

About retain cycles

To avoid a retain cycle, you should set to nil the pointer to the block when the recursion finishes.

Ramy Al Zuhouri
  • 21,580
  • 26
  • 105
  • 187
  • Sorry, I'm new to making my own blocks. I've read some things on it but i'm still a bit rusty. Can you explain that method opener a bit more? Also, it gives me an error of `block attribute not allowed, only allowed on local variables` with the __block in there. – Andrew Jul 01 '13 at 22:10
  • 1
    Ok a block is similar to a function pointer, it may be invoked just like if it's a function (*myBlock(objects)*). Every pointer referenced inside the block is copied (so the retain count of the pointed object will be increased). If instead a variable has the *__block* specifier, the pointer is not copied, but it will stay alive inside the block (similar to a global variable, but it's in the stack). Now at the moment that the block is assigned, the block variable is still *nil*, so the block referenced inside the block will be *nil*, that's why I made it have the *__block* storage specifier. – Ramy Al Zuhouri Jul 01 '13 at 22:20
  • As for the error it depends on how you're assigning the block, this error often occurs when you try to make be *__block* an ivar, have you just copied my code? – Ramy Al Zuhouri Jul 01 '13 at 22:22
  • I'm still not sure I understand what to put where or how to handle this. I've updated my post with the actual example I'm dealing with, perhaps you could help me with getting it to work for that? – Andrew Jul 01 '13 at 23:46
  • @RamyAlZuhouri There is actually a retain cycle problem with having a block recursively call itself. You can solve this one of two ways, you can set myBlock = nil; at the end of the recursion (ie. if (moreItemsNeeded == NO)), or you can use a fixed-point combinator structure. In a FPC format, the block is passed back to itself as a parameter, which can be called for recursion. Since this goes through parameter passing, you don't need to mess with __block variables and there's no risk of a retain cycle with the block variable itself. – Mr. T Jul 01 '13 at 23:55
  • @Mr.T Blocks don't truly follow common memory management rules. A retain message is ignored and thus you can't have a retain cycle in this case, the retain count will always be 1. This is made because they must be deallocated when they're out of the stack for global blocks (in the stack), or when they're not referenced for malloc blocks (copied blocks, like in this case). – Ramy Al Zuhouri Jul 02 '13 at 00:04
  • Perhaps things have changed in the more recent version of Objective-C, but it wasn't always this way. Rob Mayoff did a great analysis of this: http://stackoverflow.com/a/13091475/588253 – Mr. T Jul 02 '13 at 00:06
  • @Andrew I edited the question applying it to your code. – Ramy Al Zuhouri Jul 02 '13 at 00:20
  • @Mr.T Well the *__block* specifier explicitly says to don't retain it, so I'm pretty sure that the answer you linked is wrong. However tomorrow I'll clarify this. – Ramy Al Zuhouri Jul 02 '13 at 00:43
  • @RamyAlZuhouri Well, according to the "Transitioning to ARC Release Notes" doc provided by Apple, it says: "In manual reference counting mode, __block id x; has the effect of not retaining x. In ARC mode, __block id x; defaults to retaining x (just like all other values). To get the manual reference counting mode behavior under ARC ... options are to either use __weak..., or set the __block value to nil to break the retain cycle." (see: http://developer.apple.com/library/mac/#releasenotes/ObjectiveC/RN-TransitioningToARC/Introduction/Introduction.html) – Mr. T Jul 02 '13 at 00:50
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/32708/discussion-between-ramy-al-zuhouri-and-mr-t) – Ramy Al Zuhouri Jul 02 '13 at 00:53
4

I prefer to use the fixed-point combinator structure to write block recursion. This way I don't have to mess with __block variables or risk a retain cycle when I forget to set the block to nil at the end of the recursion. All credit for this goes to Mike Ash who shared this code snippet.

Here's my version of his code (which I placed in a globally shared file so I can access this function from anywhere):

// From Mike Ash's recursive block fixed-point-combinator strategy (https://gist.github.com/1254684)
dispatch_block_t recursiveBlockVehicle(void (^block)(dispatch_block_t recurse))
{
    // assuming ARC, so no explicit copy
    return ^{ block(recursiveBlockVehicle(block)); };
}

typedef void (^OneParameterBlock)(id parameter);
OneParameterBlock recursiveOneParameterBlockVehicle(void (^block)(OneParameterBlock recurse, id parameter))
{
    return ^(id parameter){ block(recursiveOneParameterBlockVehicle(block), parameter); };
}

I know this looks super weird and confusing... but it's not too bad once you understand it. Here's what a simple recursive block might look like:

dispatch_block_t run = recursiveBlockVehicle(^(dispatch_block_t recurse) 
{
    if (! done)
    {
        // Continue recursion
        recurse();
    }
    else
    {
        // End of recursion
    }
});
run();

When you call recursiveBlockVehicle, you're passing a block that contains your code. recursiveBlockVehicle's job is take this block that you passed and do three things:

  1. Execute the block
  2. Pass the block back through recursiveBlockVehicle and pass that resultant as the parameter to the block
  3. Encapsulate steps 1 and 2 within a simple block and return that

Now, inside your block's code, if you were to call the special recurse block parameter, you're in turn calling your own block all over again (achieving recursion). The nice thing about this strategy is that the memory management is fairly straight-forward. The use of parameters to pass your own code back to yourself reduces the risk of retain cycles. I use this method instead of defining a __block variable of my code because I'm afraid I might forget to set the __block variable to nil at the end of a recursion and result in a nasty retain cycle.

With that in mind, here's how I would implement your function:

OneParameterBlock run = recursiveOneParameterBlockVehicle(^(OneParameterBlock recurse, id parameter)
{
    NSNumber *offset = parameter;
    [noteStore
        findNotesMetadataWithFilter:filter
        offset:offset.intValue
        maxNotes:100
        resultSpec:resultSpec
        success:^(EDAMNotesMetadataList *metadataList)
        {
            for (EDAMNoteMetadata *metadata in metadataList.notes)
            {
                NSDate *timestamp = [NSDate endateFromEDAMTimestamp:metadata.updated];
                if (timestamp.timeIntervalSince1970 > date.timeIntervalSince1970)
                {
                    [array addObject:metadata];
                }
                else
                {
                    arrayComplete = YES;
                }
            }

            //I need it to loop this code, increasing the offset, until the array is complete.
            if (! arrayComplete)
            {
                recurse([NSNumber numberWithInt:offset.intValue + 100]);
            }
        }
        failure:^(NSError *error)
        {
            NSLog(@"Failure: %@", error);
        }];
});
run(@0);

Again, note that you're not calling callback (the block object) inside of the block itself. The reason why is because the block is passing itself as a parameter recurse and executing recurse is how you achieve recursion.

Also, (in case you've actually read this far and wanted to see more), here's a wikipedia page on FPC: http://en.wikipedia.org/wiki/Fixed-point_combinator

Lastly, I have not personally tested the retain cycle issue of a __block variable. However, Rob Mayoff did a fantastic analysis on the issue: https://stackoverflow.com/a/13091475/588253

Community
  • 1
  • 1
Mr. T
  • 12,795
  • 5
  • 39
  • 47
  • This worked, thank you. My only question is the purpose of `run(@0);` at the end? – Andrew Jul 02 '13 at 01:29
  • 1
    Ahh, sorry, that's a shorthand. The @0 translates to [NSNumber numberWithInt:0];. In other words, @0 translates to an NSNumber with the numerical value as 0. The purpose was to define a starting point for the offset (ie. start the offset at 0 and increment it by 100 each fetch). This can also be written as: run([NSNumber numberWithInt:0]); – Mr. T Jul 02 '13 at 02:06
2

Your code will be simpler to understand and less prone to leaking the block if you don't make the block recursive. Instead, wrap it in a method, and make the block call the method if it needs to keep searching.

This example is based on the code in your question:

- (void)appendNotesMetadataToArray:(NSMutableArray *)array
    untilDate:(NSDate *)date withFilter:(EDAMNoteFilter *)filter
    offset:(int32_t)offset resultSpec:(EDAMNotesMetadataResultSpec *)resultSpec
{
    static const int32_t kBatchSize = 100;

    [noteStore findNotesMetadataWithFilter:filter
        offset:offset maxNotes:kBatchSize resultSpec:resultSpec
        success:^(EDAMNotesMetadataList *metadataList) {
            BOOL searchComplete = NO;
            for (EDAMNoteMetadata *metadata in metadataList.notes) {
                NSDate *timestamp = [NSDate endDateFromEDAMTimestamp:metadata.updated];
                if ([timestamp compare:date] == NSOrderedDescending) {
                    [array addObject:metadata];
                } else {
                    searchComplete = YES;
                }
            }

            if (!searchComplete) {
                [self appendNotesMetadataToArray:array untilDate:date
                    withFilter:filter offset:offset + kBatchSize
                    resultSpec:resultSpec];
            }
        } failure:^(NSError *error) {
            NSLog(@"Failure: %@", error);
        }];
}

With this design, you don't need to declare a reference to the block with an inscrutable type signature, and you don't have to worry about the block leaking because it references itself.

In this design, each call to the method creates a new block. The block references self, and (I assume) self references noteStore, and noteStore references the block, so there is a retain cycle. But when the block finishes executing, noteStore releases the block, breaking the retain cycle.

rob mayoff
  • 375,296
  • 67
  • 796
  • 848
1

This is (as far as I've been able to figure out) - sort of an annoying connundrum - and one of blocks' few shortcomings... The following is the go-to archetype I refer to if I REALLY wanna make sure I'm being safe about it..

// declare a "recursive" prototype you will refer to "inside" the block.
id __block (^enumerateAndAdd_recurse)(NSArray*);        
// define the block's function - like normal.
id         (^enumerateAndAdd)        (NSArray*) = ^(NSArray*kids){ 
   id collection = CollectionClass.new;
   for (ArrayLike* littleDarling in kids) 
       [collection add:enumerateAndAdd_recurse(littleDarling)];
   return collection;
};      
enumerateAndAdd_recurse = enumerateAndAdd; // alias the block "to itself" before calling.
enumerateAndAdd(something); // kicks it all off,  yay. 
Alex Gray
  • 16,007
  • 9
  • 96
  • 118
  • I'm still not sure i understand this. How could this be adapted to fix my code (new example above)? – Andrew Jul 02 '13 at 00:16