13

I have a db set up in mongo that I'm accessing with pymongo.

I'd like to be able to pull a small set of fields into a list of dictionaries. So, something like what I get in the mongo shell when I type...

db.find({},{"variable1_of_interest":1, "variable2_of_interest":1}).limit(2).pretty()

I'd like a python statement like:

x = db.find({},{"variable1_of_interest":1, "variable2_of_interest":1})

where x is an array structure of some kind rather than a cursor---that is, instead of iterating, like:

data = []
x = db.find({},{"variable1_of_interest":1, "variable2_of_interest":1})
for i in x:
    data.append(x)

Is it possible that I could use MapReduce to bring this into a one-liner? Something like

db.find({},{"variable1_of_interest":1, "variable2_of_interest":1}).map_reduce(mapper, reducer, "data")

I intend to output this dataset to R for some analysis, but I'd like concentrate the IO in Python.

Mittenchops
  • 18,633
  • 33
  • 128
  • 246

3 Answers3

31

You don't need to call mapReduce, you just turn the cursor into a list like so:

>>> data = list(col.find({},{"a":1,"b":1,"_id":0}).limit(2))
>>> data
[{u'a': 1.0, u'b': 2.0}, {u'a': 2.0, u'b': 3.0}]

where col is your db.collection object.

But caution with large/huge result cause every thing is loaded into memory.

Nam G VU
  • 33,193
  • 69
  • 233
  • 372
Asya Kamsky
  • 41,784
  • 5
  • 109
  • 133
  • Is this a general python thing I didn't know? list() on a cursor object forces the cursor into the values its pointing to? – Mittenchops Mar 05 '13 at 14:09
  • it is a language thing - in JavaScript (that is at mongo shell you would do it like this: db.coll.find().toArray() - that turns a cursor into an array of results. – Asya Kamsky Mar 06 '13 at 23:37
  • In case one would want only one field returned from the db, is it possible to create a list of just that field like so (eg in your example for 'a'): >>> data [1.0, 2.0] ? – jaivalis Feb 08 '15 at 12:33
  • To answer my own question, the best solution I found was this: http://stackoverflow.com/questions/7271482/python-getting-a-list-of-value-from-list-of-dict – jaivalis Feb 08 '15 at 12:42
2

What you can do is to call mapReduce in pymongo and pass it the find query as an argument, it could be like this:

db.yourcollection.Map_reduce(map_function, reduce_function,query='{}')

About the projections I think that you would need to do them in the reduce function since query only specify the selection criteria as it says in the mongo documentation

lgomezma
  • 1,597
  • 2
  • 15
  • 30
0

Building off of Asya's answer: If you wanted a list of just one value in each entry as opposed to a list of objects--using a list comprehension worked for me.

I.e. if each object represents a user and the database stored their email, and you say wanted all the users that were 15 years old

user_emails = [user['email'] for user in db.people.find( {'age' : 15} )]

More here

btin
  • 306
  • 2
  • 9