2

I'm using privateGPT with the default GPT4All model (ggml-gpt4all-j-v1.3-groovy.bin) but also with the latest Falcon version. My problem is that I was expecting to get information only from the local documents and not from what the model "knows" already.

Example: If the only local document is a reference manual from a software, I was expecting privateGPT to not be able to reply to a question like: "Which is the capital of Germany?" or "What is an apple?" because it's something is not in the local document itself.

privateGPT uses a local Chroma vectorstore to store embeddings from local docs. The langchain retriever shouldn't get info only from those? what am I missing?

desertnaut
  • 57,590
  • 26
  • 140
  • 166
TheOldMan
  • 61
  • 8

1 Answers1

0

Privategpt response has 3 components (1) interpret the question (2) get the source from your local reference documents and (3) Use both the your local source documents + what it already knows to generate a response in a human like answer.

You can switch off (3) by commenting out the few lines shown below in the original code and defining

docs = db.as_retriever(search_type="similarity", search_kwargs={"k":5}).get_relevant_documents(query)

as shown below. You can adjust parameter {"k":5} to get 5 or any number of sources listed.

The draw back is if you do the above steps, privategpt will only do (1) and (2) but it will not generate the final answer in a human like response. So essentially privategpt will act like a information retriever where it will only list the relevant sources from your local documents.

Modified code

while True:
    query = input("\nEnter a query: ")
    if query == "exit":
        break

    # Get the answer from the chain
    ####res = qa(query)
    ####answer, ####
    docs = db.as_retriever(search_type="similarity", search_kwargs={"k":5}).get_relevant_documents(query)

    # # Print the result
    # print("\n\n> Question:")
    # print(query)
    # print("\n> Answer:")
    # print(answer)

    # Print the relevant sources used for the answer
    for document in docs:
        print("\n> " + document.metadata["source"] + ":")
        print(document.page_content)
Display Name is missing
  • 125
  • 9
  • Thanks but I've figure that out but it's not what i need. What I mean is that I need something closer to the behaviour the model should have if I set the prompt to something like """ Using only the following context: answer the following question: """ but it doesn't always keep the answer to the context, sometimes it answer using knowledge that is not in the provided context. – TheOldMan Sep 02 '23 at 09:05