I am currently engaged in fine-tuning the meta-llama/Llama-2-7b-chat-hf model using the Llama recipe and the LoRA technique. My approach involves employing prompt engineering to refine the model's performance, utilizing data presented in the Alpaca format:
[
{
"instruction": "What is CubeOS?",
"input": "",
"output": "CubeOS is the specialized operating system encompassing all the necessary software and drivers to operate Cubes."
},
{
"instruction": "What is Myst?",
"input": "",
"output": "Myst serves as the console interface for the Cubes, and it is also the designated name for the accompanying app."
},
.
.
.
]
This process enables me to fine-tune the model effectively and apply it to answer questions pertaining to confidential documents.
I have experimented with assigning scores to the question-answer pairs and then arranging them in order of these scores for fine-tuning. However, I have encountered challenges as the model does not seem to give results as per the higher importance of data with higher scores.
I came across an article titled https://towardsdatascience.com/how-to-fine-tune-llama2-for-python-coding-on-consumer-hardware-46942fa3cf92 where the author appears to have pursued a similar approach.
Additionally, I have explored other resources that propose a method involving the determination of relevance scores for individual features and subsequently sorting them for fine-tuning. The following link https://pyvideo.org/pydata-warsaw-2019/learning-to-rank-with-the-transformer.html provides insights into this technique.
I also attempted to train the model using the prompt structure requisite for llama-2:
<s>[INST] <<SYS>> {{ system_prompt }} <</SYS>> {{ user_message }} [/INST]
However, this approach did not yield satisfactory emphasis on the answers.
Given that I possess documents in PDF and DOC formats, my objective is to assign greater weight to specific documents and ensure their prioritized appearance as the top answers.
I would greatly appreciate your guidance on how to proceed with fine-tuning the model by incorporating weights or scores to accentuate the significance of certain documents.
