0

I am making a question answering bot with python. But for training data, I need to build a json file, collecting as much as question and answer possible from customer care websites, etc, where all the questions and answers will be stored, in the form of lists. The output json will look something like this :

{
       head : {
           questions : ["hello", "hey there", "how are you"]
           answers : ["Hey!", "Hello!", "I am good"]
         }
}

I think this method is better because the Q&A will have the same indexes, so mapping would be a lot easier from question to answer

I can ask people questions and let them answer, and all of the data should be stored in the json file. Is it possible to append items to existing json files with python?

I am thinking of something like this:

NOTE: ITS PSUEDOCODE:

import json



while True:
    question = input("enter your question: ")

    if question.lower() == 'exit':
        exit()

    answer = input("enter your answer: ")
    json.append_to_question_list(path, to="head{questions[]}", question)
    json.append_to_answer_list(path, to="head{answers[]}", answer)

I am not going for something like having the whole q&a lists in python, and then writing them once for all in json as I want to use that script later as well to keep appending my list to get a bigger data. Somthing like this was answered here: Generate a JSON file with Python

But I fear that I won't be able to add more items and use it dynamically as I expect, like in an iterative training loop, using the same index for questions and answers

Is/Are there any solution(s) to this?

Another reference I got a bit similar to this, but not sure if it satisfies the actual problem: How to dynamically build a JSON object with Python?

shakhyar.codes
  • 218
  • 4
  • 14
  • instead of using single JSON, consider using ndjson/newline-delimted json (that is regular text file where each line is valid JSON). Otherwise you will need to read whole file in memroy, add new questions and then dump back overwriting the existing file. Of course using totally different approach, like storing questions and answers in a database like sqlite3 is/may be even better. – buran Jun 08 '21 at 06:12
  • @buran if I use ndjson/newline-delimted json, will it be same like a normal json file? – shakhyar.codes Jun 08 '21 at 06:15
  • 1
    No, that's the point. each line is valid json, but whole file is not valid json. If it will work for you depends on how you want to use it (access questions) at the end. Using ndjson would allow to append new questions. for retrieveing questions you will need to read and parse the whole file anyway. – buran Jun 08 '21 at 06:15
  • For what it's worth, your example is not valid JSON; you need to quote the keywords before `:` – tripleee Jun 08 '21 at 06:18
  • @buran thanks! I will try it out, memory is not my problem anyways :-) – shakhyar.codes Jun 09 '21 at 03:08

0 Answers0