0

I'm currently trying to have an upload function on my client side that takes a csv file and uploads it into the mongoDB. If anyone has any clue on how to help I'd really appreciate it!

Here's my current code

@router.post("/uploadfile", response_description="upload a file")
async def upload(request: Request,file: UploadFile = File(...)):
    contents = file.file.read()
    buffer = BytesIO(contents)
    df = pd.read_csv(buffer)
    buffer.close()
    file.file.close()
    df.to_dict(orient='records')
    await request.app.mongodb["SentenceModel"].insert_one(file.file)

I also have gotten the ability for the file to upload into mongo but only as one document with all of the data as objects when it should be 1700 documents.. Here's that code

    @router.post("/uploadfile", response_description="upload a file")
async def upload(request: Request,file: UploadFile = File(...)):
    csvReader = csv.DictReader(codecs.iterdecode(file.file, 'utf-8'))
    data = {}
    for rows in csvReader:             
        key = rows['rowNum'] 
        data[key] = rows  
    
    file.file.close()
    await request.app.mongodb["SentenceModel"].insert_one(data)

I've tried many other methods that i've seen posted around but found no luck with any... I have no problem uploading the data but I want the data to be inserted into mongo once uploaded

Brunyan
  • 1
  • 1
  • Does this answer your question? [Insert a Pandas Dataframe into mongodb using PyMongo](https://stackoverflow.com/questions/20167194/insert-a-pandas-dataframe-into-mongodb-using-pymongo) – ray Feb 24 '23 at 01:10
  • On the second code block if I use .insert.many I get the error that can't insert empty data into db – Brunyan Feb 24 '23 at 01:14

1 Answers1

0

I fixed it! for anyone wondering how what I did wrong... I didn't put my mongo request in my for loop....

@router.post("/uploadfile", response_description="upload a file")
async def upload(request: Request,file: UploadFile = File(...)):
    data = {}
    contents = file.file.read()
    buffer = StringIO(contents.decode('utf-8'))
    csvReader = csv.DictReader(buffer)
    for row in csvReader:  
        key = row['rowNum']
        data[key] = row  
        await request.app.mongodb["SentenceModel"].insert_one(row)

    buffer.close()
    file.file.close()
Brunyan
  • 1
  • 1
  • This is performing one db call per record, which can be very costly. You may want to wrap your insertions in a bulk operation like [this](https://api.mongodb.com/python/3.2.1/examples/bulk.html) – ray Feb 24 '23 at 03:29