I am using curie model to fine-tune in Python. Basically, I am passing the training data of form {"prompt":"completion"}
and I have 736 prompt-example pairs.
My example completions are pretty long - I aim at generating a JSON file based on a description of fixed form. The fine-tune reports to be created, however, when retrieving the fine-tune model key via
retrieve_response = openai.FineTune.retrieve(id="fine_tune_model_id")
print(retrieve_response)
I get the following messsage:
{
"created_at": 1685346828,
"events": [
{
"created_at": 1685346828,
"level": "info",
"message": "Created fine-tune: fine_tune_model_id",
"object": "fine-tune-event"
},
{
"created_at": 1685346879,
"level": "info",
"message": "Error: The training file does not contain enough examples that fit within the 2048 tokens allowed for this model.",
"object": "fine-tune-event"
}
...
and thus the status failed, but for the "training files" object the status is proceeded (pretty obvious).
Is there a way to overcome the error above?
I do have an OpenAI subscription.