1

I am working with the Google Drive API, and receiving the following error when putting through watch notifications (channels) 

Error calling POST https://www.googleapis.com/drive/v2/files/xxxxxxxxxxxxxx/watch: (403) Rate limit exceeded for creating file subscriptions

Is there a specific, documented rate limited for:

a) Requesting new channels (watch notifications) per sec

b) Concurrent channels (watch notifications) per day

I have read through this question 403 rate limit after only 1 insert per second , but it doesn't really provide an answer

I would be grateful if you could point me in the right direction.

Community
  • 1
  • 1
Jamie Dan
  • 87
  • 5

2 Answers2

2

403 rate limits are your enemy - avoid them. Google does rate limiting using a token/bucket mechanism. This means you can burst a number of API calls up to around 25-39 with no problems. After that the tokens are replenished at approx 1 per second. This generally means that after the first 30, most of your subsequent transactions will fail unless you voluntarily throttle them below one every 1.5s. This is far more effective than exponential backoff, which will often result in the first backoff also failing, with success occurring on the second. The result is 3 Drive API calls with the successful call taking 3 seconds (plus network). If you throttle after the first 403, each subscription will be a single Drive API call and will succeed after 1.5s. If you are doing many subscriptions, those additional 1.5s soon add up.

One of the issues with 403's is that sometimes the transaction has actually succeeded. I doubt if it's an issue for watch subscriptions, but can be for inserts and some updates.

pinoyyid
  • 21,499
  • 14
  • 64
  • 115
1

Google does not appear to document the rate limit anywhere. They have published a document describing the recommended approach for handling API errors though.

The flow for implementing simple exponential backoff is as follows:

  1. Make a request to the API.
  2. Receive an HTTP 403 rate-limited response, which indicates you should retry the request.
  3. Wait 1 + random_number_milliseconds seconds and retry the request.
  4. Receive an HTTP 403 rate-limited response, which indicates you should retry the request.
  5. Wait 2 + random_number_milliseconds seconds, and retry the request.
  6. Receive an HTTP 403 rate-limited response, which indicates you should retry the request.
  7. Wait 4 + random_number_milliseconds seconds, and retry the request.
  8. Receive an HTTP 403 rate-limited response, which indicates you should retry the request.
  9. Wait 8 + random_number_milliseconds seconds, and retry the request.
  10. Receive an HTTP 403 rate-limited response, which indicates you should retry the request.
  11. Wait 16 + random_number_milliseconds seconds, and retry the request.
  12. Stop. Report or log an error.
abraham
  • 46,583
  • 10
  • 100
  • 152