Read and parse JSON Correctly

Question

[
    {
        u"_id": ID('1234'),
        u"attributes": {u"FN": u"John", u"LN": u"De"},
        u"refs": {
            u"entries": {
                u"id": [u"aa1", u"3da", u"42q"],
                u"type": u"Info"
            }
        }
    },
    {
        u"_id": ID('1278'),
        u"attributes": {u"FN": u"Trey", u"LN": u"Mullens"},
        u"refs": {
            u"entries": {
                u"id": [u"aa1", u"3d4", u"42q"],
                u"type": u"Info"
            }
        }
    }
]

From the above list of dicts, I want to create a function to find the count of each of the entry i.e entries under refs from the two records with ID 1234 and 1278. I am facing issues to parse the dict. Pulling data from Mongo. What I am currently trying to do is creating a function that accepts the data above, which is a collection in the mongo DB and the second argument is where I want to reach to i.e. refs.entries. I am using mongo's find function to parse the data. Any support will be highly appreciated. Below is the code -

def fun(data, tar):
    result = {}

    i, j = tar.split('.')
    for item1 in data.find({},{'_id':0, tar:1}):

            for item in item1[i][j].strip("[]").split(','):
                item = item.strip()
                if item in result:
                    result[item] += 1
                else:
                    result[item] = 1
    return result

That is not valid JSON. What is `ID('1234')`? Did you just mean dictionary? — Gillespie, Jun 09 '17 at 18:38
I am sorry @RPGillespie. I mean dict. I edited the question. — ComplexData, Jun 09 '17 at 18:41
Your question is also super vague and lacks context. "Records that exist less than 10% of the time"? What does that even mean? Do records pop in and out of existence over the lifetime of your program? — Gillespie, Jun 09 '17 at 18:41
Is this in a file? Is it already in a dict in your Python program? — aghast, Jun 09 '17 at 18:42
@RPGillespie I just mean to parse all the records, 2 records in this case with ID 1234, and 1278 and get the count of all the entries (refs.entries). Edited the question again. — ComplexData, Jun 09 '17 at 18:43
@Dreamer and presumably using a mongo library to interface with it? I'm thinking this is an XY problem and you don't want to be pulling the data down you want to execute a proper query against the data store instead. — Jon Clements, Jun 09 '17 at 18:45
@JonClements Yes Jon I guess you are correct. I am facing issues to pull the right data from the dict. Sorry for the vague language. — ComplexData, Jun 09 '17 at 18:47
@Dreamer okay - you may want to [edit] your question to more fully explain what's going on here. Explain that you're using Mongo and what library you're using to connect to it, what command you're using to get the data you already have. Mentioning Python `dict`'s and `JSON` is going to lead to people posting answers that won't help - as they're not relevant here. — Jon Clements, Jun 09 '17 at 18:48
@Dreamer okay and show your find command... (and mention what mongo library you're using etc...) - Someone will probably be able to help you get mongodb to directly return the result you want with a suitable query. — Jon Clements, Jun 09 '17 at 18:53
@JonClements Thanks for the support. I have added all details. — ComplexData, Jun 09 '17 at 18:57
That data isn't structured properly. I assume it's supposed to be a list of dicts. I'll replace it with a properly-formatted version. — PM 2Ring, Jun 09 '17 at 19:02

amrx · Accepted Answer · 2017-06-09T19:05:41.613

0

Data is a list of dicts and needs to be converted to JSON or in others way JSON encoded.

 import json

 dict_obj = [{'a': 'b', 'c': 'd', 'e': {'1': 2}}, {1: 2, 2: 3}]
 # encode to a JSON formatted str
 encoded = json.dumps(r)

 # encode to JSON formatted steam to file
 with open(filename, 'w') as f:
     json.dump(f, dict_obj)

Also the ID class objects needs to JSON serializable otherwise dump will throw an error. If your ID class is not JSON serializable, you will first have to make the ID class JSON serializable as stated here.

edited Jun 09 '17 at 19:05

answered Jun 09 '17 at 18:46

amrx

673
1
9
23

Sorry @amrx for the confusion. Its a python dict, not a JSON and I am pulling data from Mongo. – ComplexData Jun 09 '17 at 18:47
Please have a look at the edited question. Your support will be appreciated. – ComplexData Jun 09 '17 at 18:49
@Dreamer, answer changed post the question edit. Make ID class JSON serializable and do as stated in the answer. Without ID class being JSON serializable, you won't be able to do this. – amrx Jun 09 '17 at 19:00
@Dreamer, please accept the answer if it solved your query. – amrx Jun 10 '17 at 09:48

Read and parse JSON Correctly

1 Answers1