0

I have a function that will pass a string to the OpenAI API:

output = openai.Completion.create(
    model="text-davinci-003",
    # gpt-3.5-turbo-16k
    prompt=promt,
    max_tokens=4090,
    temperature=0
)

Works fine in most use cases.

In some use cases, the string sent to the API is met with the following response:

openai.error.InvalidRequestError: 
This model's maximum context length is 4097 tokens, however you requested 6227 tokens (2137 in your prompt; 4090 for the completion).

Strangely, I can go to the web interface of ChatGPT and paste in the same string sent to the API, and the web interface provides an answer - not an error.

So is there a way to either:

  1. Call gpt-3.5-turbo-16k? Since it is capable of handling 16,384 tokens.
  2. Or can is there a way to limit the max_tokens of the completion?

I'm fine with a paid version - just not finding where we pay for, say to use the gpt-3.5-turbo-16k version.

Link to max tokens per version

davos
  • 151
  • 10
  • You could tokenize on your end (https://github.com/openai/tiktoken), then request the expanded model if it's over the limit. – ceejayoz Aug 08 '23 at 18:25

0 Answers0