0

I am getting data from SQL, doing some operations and after converting it into Pandas DataFrame, finally inserting it into Mongo collection. But fields are also showing which have null values in mongo, that I don't want.

Here is dataframe:

person = [
    {
        "name" : "Tom",
        "location" : "Pune",
        "zone" : "Red",
        "profession" :"IT"

    },
    {
        "name" : "Jerry",
        "location" : "Mumbai",
        "profession":""

    }
]

df = pd.DataFrame(person)

This is how I am inserting dataframe into Mongo:

def bulkInsertData(collectionName, df):
    try:
        records = json.loads(df.to_json()).values()
        dbInst[collectionName].insert_many(records, ordered=False, bypass_document_validation=True)
    except BulkWriteError as bwe:
        print(bwe.details)
    except UnicodeEncodeError as bwe:
        print(bwe.details)

But all fields are showing. As you can see, in Mongo : 'zone' and 'profession' fields should not inserted for Jerry. Suggest some ideas.

Manish Pal
  • 292
  • 2
  • 6
  • 20

0 Answers0