Rate limit exceeded in Batch query

Question

I have client application to insert files into Google drive. One time it is required to insert multiple files into Google Drive. Batch query (GTLBatchQuery) is used to insert multiple files at a time to Google drive. Some time during insert, server is throwing rate limit exceeded error:

    "error" : {
  "message" : "Rate Limit Exceeded",
  "data" : [
    {
      "reason" : "rateLimitExceeded",
      "message" : "Rate Limit Exceeded",
      "domain" : "usageLimits"
    }
  ],
  "code" : 417
},

Please direct me correct way to enable retry on this error. I have tried setting retryenabled to service:
self.driveService.retryEnabled = YES; self.driveService.maxRetryInterval = 60.0;

But it has no effect.

1. Is is possible to set retry for Batch query?
2. Should I need to set retry enabled to GTMHTTPFetcher?

Any code snippet on Implementing exponential backoff in objective-c is appreciated.

Answer 1

Standard exponential backoff as shown in the Google documentation is not the correct way to deal with rate limit errors. You will simply overload Drive with retries and make the problem worse.

Also, sending multiple updates in a batch is almost guaranteed to trigger rate limit errors if you have more than 20 or so updates, so I wouldn't do that either.

My suggestion is:-

1. Don't use batch, or if you do, keep each batch below 20 updates
2. If you get a rate limit, backoff for at least 5 seconds before retrying
3. Try to avoid the rate limit errors by keeping your updates below 20, or keeping the submission rate below one every 2 seconds

These numbers are all undocumented and subject to change.

The reason for 3 is that there is (was, who knows) a bug in Drive that even though an update returned a rate limit error, it did actually succeed, so you can end up inserting duplicate files. See 403 rate limit on insert sometimes succeeds