0

I am working on a web app that uses generative AI (Bison model on Vertex) to answer user questions, something like an automated Q&A page.

I've fine tuned the model and I would like to submit additional information (100 Q&A) in the prompt.

Is there a way to avoid submitting all 100 Q&As at the beginning of each user session?

Actually, I don't understand how sessions/history works on Vertex project level. If I have a project that I've prompted in the past with instruction will the same project take that in consideration when answering a user question? On top of that, will user on the web be able to influence the model via prompting?

Domet
  • 1
  • 1

1 Answers1

0

If you need some extra knowledge to add to the model and it is large enough to impact to be used as context on your prompt, so it is better to have the model fine tuned with this extra information (and 100 Q&A may be a lot to be sent on all requests). Remember also that the amount of repetitive contents you send every time will be considered when billing is calculated.

About the history - no, the models behind the GenAI Studio APIs don't record conversations (only chat-bison temporarily stores for a single conversation "session" but then it is deleted too).

I didn't get what you'd meant with "will user on the web be able to influence the model via prompting" - but anything sent on a model.predict() request will be considered... So if you capture the user profile/category/preferences/comments/etc on your web app, you can embed that on the model request as extra "context".

Luciano Martins
  • 421
  • 1
  • 9