-2

I am struggling to find a way to read this json blob into tuples for each record, the author of the database wrote the data in as json blob. When I try to pull the data out of the db however I cannot find any examples on how to decode this blob data into tuples. What I shared below is the actual dump from the sqlite database which is structured as blob. I need to read that into tuples so I can use the individual values .

def query():
    dbconn = sqlite3.connect('inbox.db3')
    cursor = dbconn.cursor()
    cursor.execute("SELECT *,oid FROM inbox_v1")
    incoming_msg = (cursor.fetchall())
    print(incoming_msg)


    dbconn.commit()
    dbconn.close()



    [(10, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TERRY YES I LV MY RIG ON 24/7 ITS OUT IN THE BARN AND I JUST REMOTE INTO IT FROM MY PHONE OCCASSIONALLY. ","TO":"WA8WQU","UTC":"2020-09-25 10:43:02","_ID":"101731382614"},"type":"DELIVERED","value":""}', 10), (18, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"HIT TERRY I DONT CURRENTLY HAVE A 2M ANTENNA SETUP BUT WILL TRY TO HIT IT WITH MY HT LATER TODAY","TO":"WA8WQU","UTC":"2020-09-25 17:19:39","_ID":"101755179584"},"type":"DELIVERED","value":""}', 18), (19, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"FB ON THE TURBO MODES I DID NOT NOT KNOW THE HD ALSO INCLUDED THE MSG ID THAT IS SUPER COOL! ","TO":"WA8WQU","UTC":"2020-09-25 17:20:21","_ID":"101755221063"},"type":"DELIVERED","value":""}', 19), (27, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"I\'M DOING A LAMB DELIVERY TODAY WILL BE OFFGRID FOR A WHILE","TO":"WA8WQU","UTC":"2020-09-26 15:30:43","_ID":"101835043995"},"type":"DELIVERED","value":""}', 27), (28, '{"params":{"CMD":" MSG TO:","DIAL":3578000,"FREQ":3579117,"FROM":"WA8WQU","OFFSET":1117,"PATH":"WA8WQU","SNR":-7,"SUBMODE":0,"TDRIFT":0.20000000298023224,"TEXT":"TEST.","TO":"WA8WQU","UTC":"2020-09-27 10:41:43","_ID":"101904134116"},"type":"DELIVERED","value":""}', 28), (65, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM BRYAN, MY QRZ PAGE IS QUITE OLD NEEDS AN UPDATE HI HI NEVER NOTICED THE FEATURE OF MESSAGE ID IN THE HB ACK, VERY COOL FEATURE. HI OF 65 TODAY HERE, HV A GR8 DAY 73 DE DAN","TO":"KF1D","UTC":"2020-10-13 10:41:30","_ID":"103286490204"},"type":"DELIVERED","value":""}', 65), (66, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TERRY, A BIT STORMY LAST NIGHT! CONNECTORS ARRIVING TODAY SO I CAN CUT COAX AND GET MY 2M SSB ANT WORKING. 73 DAN","TO":"WA8WQU","UTC":"2020-10-13 10:43:31","_ID":"103286611370"},"type":"DELIVERED","value":""}', 66), (67, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TOM CHILLY 44F THIS MORN. HI OF 65F STILL GUD WRKING WEATHER! LOVE THE MESSAGE ID IN THE HB ACK FEATURE, VERY COOL! 73 DAN","TO":"K1TWH","UTC":"2020-10-13 10:48:27","_ID":"103286907291"},"type":"DELIVERED","value":""}', 67), (73, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TOM FB ON THE RAIN WE GOT A LITTLE AND NOW ITS COOLING TO MORE NORMAL FALL WEATHER. COLORS ARE AT PEAK HERE NOW. 73 DE DAN","TO":"K1TWH","UTC":"2020-10-14 11:39:52","_ID":"103376392311"},"type":"DELIVERED","value":""}', 73), (74, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TERRY HOPE ALL IS WELL ON YOUR SIDE, HOPE TODAY I CAN FIX MY COAX WOES! SMOKING RIBS FOR SUPPER, CAN\'T WAIT! 73 DE DAN. ","TO":"WA8WQU","UTC":"2020-10-14 11:41:13","_ID":"103376473008"},"type":"DELIVERED","value":""}', 74), (85, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GE TERRY, SLOWLY GETTING THROUGH THE WINTER READINESS HERE, VRY LITTLE TIME FOR RADIO BUT THOUGHT I\'D CHECK IN! 73 FOR NOW","TO":"WA8WQU","UTC":"2020-10-19 23:29:23","_ID":"103850963343"},"type":"STORE","value":""}', 85), (86, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"HI TOM, BEEN BURIED WITH WORK AND GETTING FARM READY FOR WINTER , NOT MY FAVORITE TIME OF YR! 73 FOR NOW DE DAN","TO":"K1TWH","UTC":"2020-10-19 23:31:16","_ID":"103851076419"},"type":"DELIVERED","value":""}', 86), (89, '{"params":{"CMD":" MSG ","DIAL":3578000,"FREQ":3579566,"FROM":"KD2CNC","OFFSET":1566,"PATH":"KD2CNC","SNR":-5,"SUBMODE":0,"TDRIFT":1.100000023841858,"TEXT":"GM DAN, FB ON YOUR JS8 ADD-ON! LOOKING FORWARD TO SEEING IT! TTYL 73 <DARREN","TO":"W5DMH","UTC":"2020-10-21 12:12:42","_ID":"103983240096"},"type":"UNREAD","value":""}', 89)]
Daniac21
  • 57
  • 5
  • 1
    What do you mean by "a table you can use"? What does your current code look like? See also the guidance for providing a [mre]. – tripleee Oct 25 '20 at 11:03
  • I restructured the question, I need to read blob data into tuples. – Daniac21 Oct 25 '20 at 11:17
  • Does this answer your question? [How to parse data in JSON format?](https://stackoverflow.com/questions/7771011/how-to-parse-data-in-json-format) – mkrieger1 Oct 25 '20 at 11:18
  • What you shared is a list of tuples. What is the required output? It is not clear from your question, – balderman Oct 25 '20 at 11:20
  • What I shared is the actual dump from the sqlite database which is structured as blob. I need to read that into tuples so I can use the individual values . – Daniac21 Oct 25 '20 at 11:28

1 Answers1

0

See below

import json 

data = [(10, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TERRY YES I LV MY RIG ON 24/7 ITS OUT IN THE BARN AND I JUST REMOTE INTO IT FROM MY PHONE OCCASSIONALLY. ","TO":"WA8WQU","UTC":"2020-09-25 10:43:02","_ID":"101731382614"},"type":"DELIVERED","value":""}', 10), (18, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"HIT TERRY I DONT CURRENTLY HAVE A 2M ANTENNA SETUP BUT WILL TRY TO HIT IT WITH MY HT LATER TODAY","TO":"WA8WQU","UTC":"2020-09-25 17:19:39","_ID":"101755179584"},"type":"DELIVERED","value":""}', 18), (19, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"FB ON THE TURBO MODES I DID NOT NOT KNOW THE HD ALSO INCLUDED THE MSG ID THAT IS SUPER COOL! ","TO":"WA8WQU","UTC":"2020-09-25 17:20:21","_ID":"101755221063"},"type":"DELIVERED","value":""}', 19), (27, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"I\'M DOING A LAMB DELIVERY TODAY WILL BE OFFGRID FOR A WHILE","TO":"WA8WQU","UTC":"2020-09-26 15:30:43","_ID":"101835043995"},"type":"DELIVERED","value":""}', 27), (28, '{"params":{"CMD":" MSG TO:","DIAL":3578000,"FREQ":3579117,"FROM":"WA8WQU","OFFSET":1117,"PATH":"WA8WQU","SNR":-7,"SUBMODE":0,"TDRIFT":0.20000000298023224,"TEXT":"TEST.","TO":"WA8WQU","UTC":"2020-09-27 10:41:43","_ID":"101904134116"},"type":"DELIVERED","value":""}', 28), (65, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM BRYAN, MY QRZ PAGE IS QUITE OLD NEEDS AN UPDATE HI HI NEVER NOTICED THE FEATURE OF MESSAGE ID IN THE HB ACK, VERY COOL FEATURE. HI OF 65 TODAY HERE, HV A GR8 DAY 73 DE DAN","TO":"KF1D","UTC":"2020-10-13 10:41:30","_ID":"103286490204"},"type":"DELIVERED","value":""}', 65), (66, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TERRY, A BIT STORMY LAST NIGHT! CONNECTORS ARRIVING TODAY SO I CAN CUT COAX AND GET MY 2M SSB ANT WORKING. 73 DAN","TO":"WA8WQU","UTC":"2020-10-13 10:43:31","_ID":"103286611370"},"type":"DELIVERED","value":""}', 66), (67, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TOM CHILLY 44F THIS MORN. HI OF 65F STILL GUD WRKING WEATHER! LOVE THE MESSAGE ID IN THE HB ACK FEATURE, VERY COOL! 73 DAN","TO":"K1TWH","UTC":"2020-10-13 10:48:27","_ID":"103286907291"},"type":"DELIVERED","value":""}', 67), (73, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TOM FB ON THE RAIN WE GOT A LITTLE AND NOW ITS COOLING TO MORE NORMAL FALL WEATHER. COLORS ARE AT PEAK HERE NOW. 73 DE DAN","TO":"K1TWH","UTC":"2020-10-14 11:39:52","_ID":"103376392311"},"type":"DELIVERED","value":""}', 73), (74, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GM TERRY HOPE ALL IS WELL ON YOUR SIDE, HOPE TODAY I CAN FIX MY COAX WOES! SMOKING RIBS FOR SUPPER, CAN\'T WAIT! 73 DE DAN. ","TO":"WA8WQU","UTC":"2020-10-14 11:41:13","_ID":"103376473008"},"type":"DELIVERED","value":""}', 74), (85, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"GE TERRY, SLOWLY GETTING THROUGH THE WINTER READINESS HERE, VRY LITTLE TIME FOR RADIO BUT THOUGHT I\'D CHECK IN! 73 FOR NOW","TO":"WA8WQU","UTC":"2020-10-19 23:29:23","_ID":"103850963343"},"type":"STORE","value":""}', 85), (86, '{"params":{"CMD":" MSG ","DIAL":0,"FREQ":0,"FROM":"W5DMH","OFFSET":0,"PATH":"W5DMH","SNR":0,"SUBMODE":0,"TDRIFT":0,"TEXT":"HI TOM, BEEN BURIED WITH WORK AND GETTING FARM READY FOR WINTER , NOT MY FAVORITE TIME OF YR! 73 FOR NOW DE DAN","TO":"K1TWH","UTC":"2020-10-19 23:31:16","_ID":"103851076419"},"type":"DELIVERED","value":""}', 86), (89, '{"params":{"CMD":" MSG ","DIAL":3578000,"FREQ":3579566,"FROM":"KD2CNC","OFFSET":1566,"PATH":"KD2CNC","SNR":-5,"SUBMODE":0,"TDRIFT":1.100000023841858,"TEXT":"GM DAN, FB ON YOUR JS8 ADD-ON! LOOKING FORWARD TO SEEING IT! TTYL 73 <DARREN","TO":"W5DMH","UTC":"2020-10-21 12:12:42","_ID":"103983240096"},"type":"UNREAD","value":""}', 89)]

data1 =[(d[0],json.loads(d[1]),d[2]) for d in data]
print(data1[0])

another option - create a dict

data2 ={d[0]: json.loads(d[1]) for d in data}
print(data2[89])
balderman
  • 22,927
  • 7
  • 34
  • 52
  • That breaks the blob into records which is big jump in the right direction. I can use the index to pull each record but now how do I index each value in the record? – Daniac21 Oct 25 '20 at 11:40
  • Not sure i understand what is the index and I want want to use it. Explain what is the data struct u r looking for.See the updated code. – balderman Oct 25 '20 at 11:41
  • So in that dataI need to check if the record PATH value is @MPURN, if that is true I will create a tuple of FROM value, TEXT value, and UTC value. this data is then used to populate a new table. – Daniac21 Oct 25 '20 at 11:49