Memory issue generated in iPad because data receiving very lengthy

Question

I am struck in a big problem. What I am trying to do is getting data from the server but the server in a single hit give all the data which is very large in quantity and I am doing all my process on the main thread, So there are around 400-500 images in the form of URL which I am saving in document directory in the form of NSData. So in the dubug navigator when the memory consumption reached around 80-90 mb then my application crashed and showing the following error:-

mach_vm_map(size=135168) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
2015-01-23 17:10:03.946 ArchaioMobileViewer[853:148470] *** Terminating app due to uncaught exception 'NSMallocException', reason: 'Attempt to allocate 262144 bytes for NS/CFData failed'

I am Using ARC but still I am getting the memory problem. This is my code `-

(void)downloadDocumentsFromServer:(NSDictionary *)documentsList IsUpdate:(BOOL)isUpdate;
{
    //Main Target(22)


BusinessLayer* bLL = [[BusinessLayer alloc]init];
FileManager* downloadImages = [FileManager alloc];

for(NSDictionary* inspDocumentResult in documentsList)
{

    FloorDocument* floorDocument = [[FloorDocument alloc]init];
    floorDocument.docID = [inspDocumentResult objectForKey:@"docID"];
    floorDocument.buildingID = selectedBuildingID;
    floorDocument.clientID = clientID;

    NSDictionary* documentArray = [inspDocumentResult objectForKey:@"Document"];

    floorDocument.docType = [[documentArray objectForKey:@"Type"] stringByTrimmingCharactersInSet:
                             [NSCharacterSet whitespaceAndNewlineCharacterSet]];
    floorDocument.docScale = [documentArray objectForKey:@"Scale"];
    floorDocument.docDescription = [documentArray objectForKey:@"DocDesc"];
    //floorDocument.floor = [bLL getFloorNameByDocIDAndBuildingID:selectedBuildingID DocID:floorDocument.docID];
    floorDocument.floor = [inspDocumentResult objectForKey:@"Floor"];

    NSLog(@"%@",[inspDocumentResult objectForKey:@"hiResImage"]);

    [downloadImages downloadInspectionDocuments:[inspDocumentResult objectForKey:@"hiResImage"] ImageName:floorDocument.docID FileType:floorDocument.docType Folder:selectedBuildingID];
    NSLog(@"Floor %@ - High Res Image copied for %@",floorDocument.floor,floorDocument.docID);

    //Download the Low Res Image
    NSString* lowResImage = [inspDocumentResult objectForKey:@"lowResImage"];

    [downloadImages downloadInspectionDocumentsLowRes:lowResImage ImageName:floorDocument.docID FileType:floorDocument.docType Folder:selectedBuildingID LowResName:@"lowResImage"];

    //Copy the Quarter Size File
    lowResImage = [lowResImage stringByReplacingOccurrencesOfString:@"LowRes" withString:@"LowRes4"];
    [downloadImages downloadInspectionDocumentsLowRes:lowResImage ImageName:floorDocument.docID FileType:floorDocument.docType Folder:selectedBuildingID LowResName:@"lowResImage4"];


    NSLog(@"Floor %@ - Low Res Images copied for %@",floorDocument.floor,floorDocument.docID);

    //Download the tiles
    NSArray* tiles = [inspDocumentResult objectForKey:@"lsUrls"];

    for(NSString* tile in tiles)
    {
        @autoreleasepool {
            NSArray* tileNameArray = [tile componentsSeparatedByString:@"/"];

            if(tileNameArray.count > 0)
            {
                NSString* destTile = [tileNameArray objectAtIndex:tileNameArray.count-1];
                destTile = [destTile stringByReplacingOccurrencesOfString:[NSString stringWithFormat:@".%@",floorDocument.docType] withString:@""]; 

                NSLog(@"TileName:%@",destTile);
                [downloadImages downloadInspectionDocumentsTiles:tile ImageName:floorDocument.docID FileType:floorDocument.docType Folder:selectedBuildingID TileName:destTile];  
            }
        }


    }

    NSLog(@"Floor %@ - Tiles Image copied for %@",floorDocument.floor,floorDocument.docID);
    NSLog(@"Downloading Documents Tiles For %@ Completed at %@",floorDocument.docID,[bLL getCurrentDate]);
    [bLL saveFloorDocuments:floorDocument IsUpdate:isUpdate];
    // downloadImages=nil;


}

bLL = nil;

} please help me out in this problem.`

This is the code which I am using inside the DownloadInspectionDocuments:-

    -(void)downloadInspectionDocuments:(NSString *)url ImageName:(NSString *)imageName FileType:(NSString*)fileType Folder:(NSString*)folder
{
    @autoreleasepool
  {
    NSString* source =[FileManager getInspectionDocumentsFolder];
    //Lets get the destination folder
    NSString *destination = [NSString stringWithFormat:@"%@/%@/%@",source,folder,imageName];

    [self createFolder:destination CreateSubFolders:true];

    NSString *filePath = [NSString stringWithFormat:@"%@/%@.%@",destination,imageName,fileType];
    NSFileManager* fm = [[NSFileManager alloc]init];

        if(![fm fileExistsAtPath:filePath])
        {
            NSData *data1 = [NSData dataWithContentsOfURL:[NSURL URLWithString:url]];
            [data1 writeToFile:filePath atomically:YES];
        }

    }

   // return [NSString stringWithFormat:@"%@.%@",imageName,fileType]; 
}

possible duplicate of [double free\*\*\* set a breakpoint in malloc\_error\_break to debug in ARC](http://stackoverflow.com/questions/10587266/double-free-set-a-breakpoint-in-malloc-error-break-to-debug-in-arc) — iOSNoob, Jan 23 '15 at 12:03
If your problem is that you download huge NSData object, it doesn't make sense making this object floating around in the memory. Using NSFileHandle, NSStream or (better) AFnetworking you can append pieces of the NSData in a file on the file system. — Andrea, Jan 23 '15 at 12:49
As @Andrea says, this code is very expensive in memory use. Better to write to disk as the data arrives from the network than to stage the whole thing in memory. — Paulw11, Jan 23 '15 at 12:54

score 0 · Answer 1 · answered Jan 23 '15 at 13:00

ARC isn't garbage collection: it will insert the memory management code (retain/release) for you but you still need to make sure you are not using up too many resources (in the same way you would in non-ARC code).

You are running this large loop on the main thread, so any memory being consumed is not going to be freed until the next run loop.

You need to break this function down into smaller steps that can be carried out in stages.

For now, if there isn't too much memory consumed for a single iteration of the outer-loop of the function you can add an autorelease pool at that level (I see you have on on the inner loop)

for(NSDictionary* inspDocumentResult in documentsList)
{
 @autoreleasepool {
.... remaining code goes here
}
}

and it will at least drain what it can each iteration.

Given that you are downloading a large number of files and will be relying on network connectivity I would recommend performing the downloads asynchronously though. If you haven't already, check out AFNetworking to simplify this. This will give you much more control over your resources than you are getting now with a resource-intensive blocking call on the main thread.

but i already have used autorelease pool from where i call the method. so it is equivalent to your suggestion. Now I am going to break the function then see what exactly happening. — Abhishek Rajput, Jan 23 '15 at 13:06

score 0 · Answer 2 · answered Jan 23 '15 at 13:51

You can save yourself a lot of work by following davbryn's and Andrea's suggestions to use AFNetworking and stream the file. Basically, don't put the whole file in memory and then write it to disk, write to disk as you get the bytes from the network. This should reduce the pressure on memory. For example:

- (void)downloadFile:(NSString *)urlString {
  NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:urlString]];
  NSString *destinationPath = [NSDocumentDirectory() stringByAppendingPathComponent:@"some-file-name"];

  AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
  [operation setOutputStream:[NSOutputStream outputStreamToFileAtPath:destinationPath append:NO]]; 
  [operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) { 
    NSLog(@"Super duper awesome!");
    // Maybe start another download here?
  } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
    NSLog(@"Error downloading file: %@", error);
  }];

  [operation start];
}

So then all you need to do is generate the list of things to download and in your success block start downloading another file.

You may be right but the thing is I already have made my app completed. So I don't want to rework just because of this issue. — Abhishek Rajput, Jan 23 '15 at 14:06
This is a serious issue though, one serious enough to hinder the usefulness of your app. Sometimes it's hard to let go of the path you have already walked down, but it could be best to turn around and go back. Another option is to look at what AFNetwork is doing to stream to the file and replicate that yourself. — Scott Grant, Jan 23 '15 at 14:07

Memory issue generated in iPad because data receiving very lengthy

2 Answers2