How to parse json optimal way?

Asked Apr 22 '17 at 14:36

Active Apr 11 '20 at 15:14

Viewed 227 times

I have a JSON document with 8kk objects that looks like this

{'_id': ObjectId('55d6e5114d02f49f998a08b5'),
 'created_at': 1440146705.612493,
 'message': 'Ð\x9fÑ\x80Ð¸Ð²ÐµÑ\x82! Ð\x97Ð´ÐµÑ\x81Ñ\x8c Ð²Ñ\x8b Ð¼Ð¾Ð¶ÐµÑ\x82Ðµ Ð·Ð°Ð´Ð°Ñ\x82Ñ\x8c Ð²Ð¾Ð¿Ñ\x80Ð¾Ñ\x81 Ð¸ Ð±Ñ\x8bÑ\x81Ñ\x82Ñ\x80Ð¾ Ð¿Ð¾Ð»Ñ\x83Ñ\x87Ð¸Ñ\x82Ñ\x8c Ð¾Ñ\x82Ð²ÐµÑ\x82 Ð½Ð° Ð½ÐµÐ³Ð¾.',
 'room': '55d5928a4d02f4a55007344c',
 'sender': '55af71054d02f4571abe1f1f'}

I want to parse it and create a data frame. So I create a data frame and then concat it to already existing one so I do

import pandas as pd


df = pd.DataFrame(pd.Series(my_json[0],index=my_json[0].keys())).transpose()
for i in my_json:
    df = pd.concat([df,pd.DataFrame(pd.Series(i,index=i.keys())).transpose()])

But it takes too long. Is there any other proper way to do so?

_id 55db1fff4d02f4aeb974dc6e
message Привет! Здесь вы можете задать вопрос и быстро...
room    55db1ffe4d02f4aeb974dc6c
created_at  1.44042e+09
sender  55af71054d02f4571abe1f1f

Want to get something like that but transposed

edited Apr 11 '20 at 15:14

Martin Gergov

1,556
4
20
29

asked Apr 22 '17 at 14:36

Vladimir Ershov

Pandas has a function for loading data from json: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_json.html – Styrke Apr 22 '17 at 14:40
It didn't work for me because it is actually bson – Vladimir Ershov Apr 22 '17 at 14:46
Maybe help [this](http://stackoverflow.com/a/16255680/2901002). – jezrael Apr 22 '17 at 15:05

How to parse json optimal way?

0 Answers0