1

I was observirng some strange behaviour of my app sometime caching responses and sometime not caching them (all the responses have Cache-Control: max-age=600).

The test is simple: I did a test.php script that was just setting the headers and returning a simple JSON:

<?php
        header('Content-Type: application/json');
header('Cache-Control: max-age=600');
?>
{
    "result": {
        "employeeId": "<?php echo $_GET['eId']; ?>",
                "dateTime": "<?php echo date('Y-m-d H:i:s'); ?>'" }
}

This is the response I get from the PHP page:

HTTP/1.1 200 OK
Date: Thu, 28 Nov 2013 11:41:55 GMT
Server: Apache
X-Powered-By: PHP/5.3.17
Cache-Control: max-age=600
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json

{
    "result": {
        "employeeId": "",
        "dateTime": "2013-11-28 11:41:55'" 
    }
}

Then I've created a simple app and added AFNetworking library.

When I call the script with few parameters, the cache works properly:

AFHTTPRequestOperationManager *manager = [AFHTTPRequestOperationManager manager];

NSDictionary *params = @{ 
                         @"oId": @"4011",
                         @"eId": self.firstTest ? @"1" : @"0",
                         @"status": @"2031",
                         };
[manager GET:@"http://www.mydomain.co.uk/test.php" parameters:params success:^(AFHTTPRequestOperation *operation, id responseObject) {
    NSLog(@"JSON: %@", responseObject);

    NSLog(@"Cache current memory usage (after call): %d", [cache currentMemoryUsage]);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
    NSLog(@"Error: %@", error);
}];

But when I increase the number of parameters, like:

NSDictionary *params = @{
                         @"organizationId": @"4011",
                         @"organizationId2": @"4012",
                         @"organizationId3": @"4013",
                         @"organizationId4": @"4014",
                         @"organizationId5": @"4015",
                         @"organizationId6": @"4016",
                         @"eId": self.firstTest ? @"1" : @"0",
                         @"status": @"2031",
                         };

it doesn't work anymore and it execute a new request each time it is called.

I've done many tests and it seems to me that it is related to the length of the URL, because if I includes this set of params:

NSDictionary *params = @{
                         @"oId": @"4011",
                         @"oId2": @"4012",
                         @"oId3": @"4013",
                         @"oId4": @"4014",
                         @"oId5": @"4015",
                         @"oId6": @"4016",
                         @"eId": self.firstTest ? @"1" : @"0",
                         @"status": @"2031",
                         };

It works!!

I've done many tests and that's the only pattern I've found...

To exclude AFNetworking from the equation, I've created another test program that uses NSURLConnection only and I can see the same behaviour so it's not AFNetworking and definitely NSURLCache. This is the other test:

NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@&organizationId=4011&organizationId2=4012&organizationId3=4013&organizationId4=4014&organizationId5=4015&organizationId6=4016", self.firstTest ? @"1" : @"0"]];  // doesn't work
//NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@&oId=4011&oId2=4012&oId3=4013&oId4=4014&oId5=4015&oId6=4016", self.firstTest ? @"1" : @"0"]];  // work
//NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@", self.firstTest ? @"1" : @"0"]];  // work

NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSURLResponse *response = nil;
NSError *error = nil;
NSData *data = [NSURLConnection sendSynchronousRequest:request
                                     returningResponse:&response
                                                 error:&error];

if (error == nil) {
    // Parse data here
    NSString *responseDataStr = [NSString stringWithUTF8String:[data bytes]];
    NSLog(@"Response data: %@", responseDataStr);
}

I've also tried to establish how many characters in the URL will trigger the problem but even in this case I've got strange results:

This one is 112 characters long and it doesn't work:

http://www.mydomain.co.uk/test.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&orgaId4=4

This one is 111 characters long and it works:

http://www.mydomain.co.uk/test.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&orgId4=4

Ive renamed the PHP script to see if the first part of the URL would matter and I've got a strange behaviour again:

This one is 106 characters long and it doesn't work:

http://www.mydomain.co.uk/t.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&org=40

This one is 105 characters long and it works:

http://www.mydomain.co.uk/t.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&org=4

So I've removed 3 characters from the page name and I've got a working threshold 6 characters lower.

Any suggestion?

Thanks, Dem

demetrio812
  • 290
  • 3
  • 15

3 Answers3

4

I am witnessing something similar with certain responses not being cached by NSURLCache and I have come up with another possible reason:

In my case I have been able to ascertain that the responses not being cached are the ones that are returned using Chunked transfer-encoding. I've read elsewhere that NSURLCache should cache those after iOS 6 but for some reason it doesn't in my case (iOS 7.1 and 8.1).

I see that your example response shown here, also has the Transfer-Encoding: chunked header.

Could it be that some of your responses are returned with chunked encoding (those that are not cached) and some are not (those that are cached)?

My back-end is also running PHP on Apache and I still can't figure out why it does that... Probably some Apache extension...

Anyway, I think it sounds more plausible than the request URL length scenario.


EDIT:

It's been a while, but I can finally confirm that in our case, it is the chunked transfer encoding that causes the response not to be cached. I have tested that with iOS 7.1, 8.1, 8.3 and 8.4.

Since I understand that it is not always easy to change that setting on your server, I have a solution to suggest, for people who are using AFNetworking 2 and subclassing AFHTTPSessionManager.

You could add your sub-class as an observer for AFNetworking's AFNetworkingTaskDidCompleteNotification, which contains all the things you will need to cache the responses yourself. That means: the session data task, the response object and the response data before it has been processed by the response serializer.

If your server uses chunked encoding for only a few of its responses, you could add code in -(void)didCompleteTask: to only cache responses selectively. So for example you could check for the transfer-encoding response header, or cache the response based on other criteria.

The example HTTPSessionManager sub-class below caches all responses that return any data:

MyHTTPSessionManager.h

@interface MyHTTPSessionManager : AFHTTPSessionManager


@end

MyHTTPSessionManager.m

#import "MyHTTPSessionManager.h"

@implementation MyHTTPSessionManager

+ (instancetype)sharedClient {
    static MyHTTPClient *_sharedClient = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        [[NSNotificationCenter defaultCenter] addObserver:_sharedClient selector:@selector(didCompleteTask:) name:AFNetworkingTaskDidCompleteNotification object:nil];
    });

    return _sharedClient;
}

- (void)didCompleteTask:(NSNotification *)notification {
    NSURLSessionDataTask *task = notification.object;
    NSHTTPURLResponse *response = (NSHTTPURLResponse *)task.response;

    NSData *responseData = notification.userInfo[AFNetworkingTaskDidCompleteResponseDataKey];
    if (!responseData.length) {
        // Do not cache empty responses.
        // You could place additional checks above to cache responses selectively.
        return;
    }

    NSCachedURLResponse *cachedResponse = [[NSCachedURLResponse alloc] initWithResponse:response data:responseData];
    [[NSURLCache sharedURLCache] storeCachedResponse:cachedResponse forRequest:task.currentRequest];
}

I tried to come up with some sort of cleaner solution, but it seems that AFNetworking does not provide a callback or a delegate method that returns everything we need early enough - that is, before it has been serialized by the response serializer.

Hope people will find this helpful :)

m_katsifarakis
  • 1,777
  • 1
  • 21
  • 27
  • 1
    We are seeing this same issue on our end! Did you end up resolving this, by chance? Is your assumption that there is a bug in NSURLCache when handling a response with `transfer-encoding: chunked`? – readyornot Mar 18 '15 at 01:11
  • Unfortunately we did not have enough time to test it more on iOS, so we ended up disabling chunked responses in Apache. Here's a relevant post: http://stackoverflow.com/questions/10300446/server-sometimes-returns-chunked-transfer-encoding I did not have enough time to test if chunked transfers are the only reason why NSURLCache breaks. Perhaps it's that, combined with gzip compression (which we also use). I will do some more tests when I have time and post the results here. – m_katsifarakis Mar 18 '15 at 13:37
  • Hi goatrance, any update on this one? Should NSURLCache handle chunked transfers? I'm facing the same problem, thanks for sharing. – Hai Hw Mar 30 '15 at 03:24
  • I can finally confirm that in our case, what prevents responses from being cached, is the chunked transfer-encoding. Sorry for taking so long to respond to this. I could not find anything in the Apple docs, but I have tested it thoroughly by disabling gzip compression and turning chunked transfer-encoding on and off. – m_katsifarakis Aug 31 '15 at 08:03
0

Did you try to configure NSURLRequestCachePolicy for NSURLRequest

+ (id)requestWithURL:(NSURL *)theURL cachePolicy:(NSURLRequestCachePolicy)cachePolicy timeoutInterval:(NSTimeInterval)timeoutInterval

These constants are used to specify interaction with the cached responses.

enum
{
 NSURLRequestUseProtocolCachePolicy = 0,
 NSURLRequestReloadIgnoringLocalCacheData = 1,
 NSURLRequestReloadIgnoringLocalAndRemoteCacheData =4,
 NSURLRequestReloadIgnoringCacheData = NSURLRequestReloadIgnoringLocalCacheData,
 NSURLRequestReturnCacheDataElseLoad = 2,
 NSURLRequestReturnCacheDataDontLoad = 3,
 NSURLRequestReloadRevalidatingCacheData = 5
};
typedef NSUInteger NSURLRequestCachePolicy;
Eugene P
  • 554
  • 4
  • 19
0

You could investigate what your cached response is from the sharedURLCache by subclassing NSURLProtocol and overriding startLoading:

add in AppDelegate application:didFinishLaunchingWithOptions:

[NSURLProtocol registerClass:[CustomURLProtocol class]];

Then create a subclass of NSURLProtocol (CustomURLProtol) and override startLoading

- (void)startLoading
{

self.cachedResponse = [[NSURLCache sharedURLCache] cachedResponseForRequest:self.request];

if (self.cachedResponse) {
    [self.client URLProtocol:self
          didReceiveResponse:[self.cachedResponse response]
          cacheStoragePolicy:[self.cachedResponse storagePolicy]];
    [self.client URLProtocol:self didLoadData:[self.cachedResponse data]];
}
[self.client URLProtocolDidFinishLoading:self];
}

self.cachedResponse is a property NSCachedURLResponse i've added. You can see if anything is wrong with any cachedResponse here.

Matt J
  • 58
  • 5