How to overcome Rate limit error while working with GPT3 Models using Tenacity

Question

In my situation I am trying to pass a prompt using a helper function to the actual GPT3 models, in my case text-ada-001 and then eventually applying it on a pandas column using the following code. but I am recovering the following error:

    def sentiment_prompt(text):
    return """Is the sentiment Positive, Negative or Neutral for the following text:
    
    "{}"
    """.format(text)
    def sentiment_text(text):
        response = openai.Completion.create(
           engine="text-ada-001",
           prompt=sentiment_prompt(text),
           max_tokens=1000,
           temperature=0,
           top_p=1,
           frequency_penalty=0,
           presence_penalty=0
    )
    sentiment = response.choices[0].text
    return sentiment

and then eventually applying to my pandas column:

    df['sentiment'] = df['text'].apply(lambda x :sentiment_text(x))

And the error;

    RateLimitError: Rate limit reached for default-global-with-image-limits in organization org-XXXX on requests per min. Limit: 60 / min. Please try again in 1s. Contact support@openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.

To overcome this error I was looking into this link and found that tenacity could help resolve my issue. But I am not sure how to structure my code. I am doing the following at the moment

How do I use the code suggested in the link to overcome the Rate Limit error?

Check out the retry module and wrap your sentiment_text in a retry with an incrementing retry wiat time. You can't get around rate limits but you can handle/retry the requests more robustly — rayad, Mar 31 '23 at 23:52

score 1 · Accepted Answer · edited May 28 '23 at 11:16

Import tenacity at the beginning of your code and then add its decoration where you are calling the OpenAI library with create. So your code would look like this:

from tenacity import (
    retry,
    stop_after_attempt,
    wait_random_exponential,
) 

@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))
def sentiment_text(text):
        your_prompt = """Is the sentiment Positive, Negative or Neutral for the 
                         following text:
    
                         "{}"
                      """.format(text)
        response = openai.Completion.create(
           engine="text-ada-001",
           prompt=your_prompt ,
           max_tokens=1000,
           temperature=0,
           top_p=1,
           frequency_penalty=0,
           presence_penalty=0
        )
        sentiment = response.choices[0].text
        return sentiment

How to overcome Rate limit error while working with GPT3 Models using Tenacity

1 Answers1