0

I have the following problem. I am getting some data from an api in form of a json.
I want to load this json data into an existing table.

I get the data in following format:

{
  "abcd": "blablabla",
  "value": [
    {
        "a": "1",
        "b": "2",
        "c": "3"
    },
    {
        "a": "4",
        "b": "5",
        "c": "6"
    }
  ]
}

I am only interested in the data in "value" with "a", "b" and "c" being my columns in the table.
I get only the interesting values by creating a pandas Dataframe:

contents = pandas.json_normalize(response.json(), record_path=['value'])

I can insert this data into mssql with the to_sql method from pandas, however, I would like to do an insert if it doesn't exist or update if a certain key already exists. (e.g. in this example column "a" could be the key column).

I have read about certain sqlalchemy methods with merge and so on, however I was no able to reproduce it in my case.

Here is how I loaded the data into my database:

with open('db_con.json') as db:
            con = json.load(db)

            con_url = URL.create(
                "mssql+pyodbc",
                username = con['username'],
                password = con['password'],
                host = con['host'],
                port = con['port'],
                database = con['database'],
                query = con['query'],
                )
            engine = sqlalchemy.create_engine(con_url, fast_executemany = False, echo = True)

            contents.to_sql(
                test_db,
                engine,
                if_exists='append',
                index = False,
                chunksize = 2500,
                schema = 'test'
            )

What would I need to do, to implement such an upsert method?
I guess I cannot use to_sql anymore.

Thom A
  • 88,727
  • 11
  • 45
  • 75
fnavw
  • 1

0 Answers0