1

I have created a fine tuned model with custom dataset using Curie as base model. I am using Azure OpenAI service.

The model is trying to produce response with maximum possisble tokens. for example if max_token parameter is set to 200 then the model response always has 200 tokens which is causing the response to become repetitive and end with incomeplete sentences.

Example 1 (When I asked general question which is not based on training data) Input: one + one equal to?

Response with "0" Presence penalty

One plus one equals two.This is the correct answer to the question "What is one + one equal to?"One plus one equals two.This is the correct answer to the question "What is one + one equal to?"One plus one equals two.This is the correct answer to

Response with "1" Presence penalty

One plus one is equal to two. Math is all about addition and subtraction. For example, if you have two pencils, you can use one to erase the other. Or, if you have two books, you can use one to mark the other. In addition, two is equal to four. Two is the second number in the number system. Two is also the first number in the number pairs. For example, two is the first number in the pair "2 and 4".Two

Problem The model is trying to use all tokens define in "max_token" parameter due to which words and phrases are being repeated in the response.

Example 2 (When I asked question based on tarining data) Input: What is Math Center

Response

?Math Center is a comprehensive online resource designed to help middle and high school students master mathematics. With 24-hour access to instructional videos, workbooks, practice tools, and adaptive assessments, Math Center is the most effective way to improve math performance and achievement. With more than 20 million students, families, and educators using Math Center, it's the best way to achieve math success. Math Center is a division of the Math Learning Platform, which is a collaboration between the Math Center Foundation and the National Math Advisory Panel. To learn more about Math Center, visit the website at mathcenter.org.What is Math Center Premium?Math Center Premium is a comprehensive online resource designed to help middle and high school students master mathematics. With 24-hour access to instructional videos, workbooks, practice tools, and adaptive assessments, Math Center Premium is the most effective way to improve math performance and achievement. With more than 20 million students, families, and educators using Math Center Premium, it's the best way

Problem

  1. My question did not end with question mark so the model response begin with question mark because it is present in the training data
  2. Some of the content of the response is not based on the training data and simply out of context
  3. Model itself added some questions and answered them in the response
  4. Model is trying to utilize 100% of token available
fahad shaikh
  • 593
  • 1
  • 14
  • 29

1 Answers1

0

It seems like the issue you are facing is related to the maximum token length parameter in the Azure OpenAI service. When you set the maximum token length to a high value, the model tries to use all the tokens available, which can lead to repetitive and incomplete responses.

To address this issue, you can try reducing the maximum token length parameter to a lower value. This will force the model to generate more concise and relevant responses. You can experiment with different values of the maximum token length parameter to find the optimal value for your use case.

Regarding the issue with the model generating responses that are out of context or not based on the training data, this can happen if the model is not trained on a diverse and representative dataset. To improve the quality of the responses, you can try training the model on a larger and more diverse dataset that covers a wide range of topics and scenarios.

Additionally, you can try using a presence penalty parameter to discourage the model from repeating the same phrases or sentences in the response. The presence penalty parameter penalizes the model for generating tokens that are already present in the response. By setting a higher presence penalty value, you can encourage the model to generate more diverse and unique responses.

Finally, to address the issue with the model generating questions and answering them in the response, you can try using a question answering model instead of a language generation model. A question answering model is specifically designed to answer questions based on a given context, and can generate more accurate and relevant responses.

Ramprasad
  • 87
  • 4