python ijson large file loop to get names

Question

I need to parse a large json file with ijson(unless something is better), I want to loop through all of the product names in the request and print them out. I tried to set this up using this support page. https://pypi.python.org/pypi/ijson/

This is the current output that I get

<addinfourl at 140643118020800 whose fp = <socket._fileobject object at 0x7fea07882850>>
<generator object items at 0x7fea077dc910>
<generator object <genexpr> at 0x7fea077dc960>

My code

import json
import requests
import lxml 
import ijson
import urllib
from urllib import urlopen


request = urlopen('www.jsonurl.com')
objects = ijson.items(request, 'items.name')
products = (o for o in objects if o ['type' == 'name'])
for product in products:
    print product

print request
print objects
print products

Here is a piece of the json data

{"query":"*","sort":"relevance","responseGroup":"base","totalResults":5158058,"start":1,"numItems":10,"items":[{"itemId":7933617,"parentItemId":7933617,"name":"Nordic Ware Heavyweight Scone / Cornbread Pan","msrp":26.97,"salePrice":20.42,"upc":"011172016409","categoryPath":"Home/Kitchen & Dining/Cookware, Bakeware & Tools/Specialty Cookware","shortDescription":"<p>This Nordic Ware Scone Pan is made of a heavyweight cast aluminum. It can be used as a heavyweight scone or cornbread pan, and it is designed to cook your meal evenly and thoroughly. It features a non-stick interior coating for easy release and clean up.</p>","longDescription":"<b>Nordic Ware Heavyweight Scone/Cornbread Pan:</b><ul><li>Heavyweight cast aluminum</li><li>Ideal for scones and cornbread</li><li>Eight wedges</li><li>Cooks evenly and thoroughly</li><li>Non-stick interior coating for easy release and clean-up</li></ul>","thumbnailImage":"http://i5.walmartimages.com/dfw/dce07b8c-c739/k2-_6fb32a28-c090-4377-81d5-e83273124841.v1.jpg","mediumImage":"http://i5.walmartimages.com/dfw/dce07b8c-ddb3/k2-_6f7df9fa-cb2d-4faf-afbc-8fa4185add59.v1.jpg","largeImage":"http://i5.walmartimages.com/dfw/dce07b8c-5bd3/k2-_6635f62a-5e0b-4c4e-a93d-ee85643f7397.v1.jpg","productTrackingUrl":"http://linksynergy.walmart.com/fs-bin/click?id=|LSNID|&offerid=223073.7200&type=14&catid=8&subid=0&hid=7200&tmpid=1082&RD_PARM1=http%253A%252F%252Fwww.walmart.com%252Fip%252FNordicWare-Heavyweight-Scone-Cornbread-Pan%252F7933617%253Faffp1%253DpjiPu5Y7cvNmz4xZOAs5j7QlW2mZPVmc1DR3BvmrkB4%2526affilsrc%253Dapi","standardShipRate":4.97,"marketplace":false,"modelNumber":"1640","productUrl":"http://c.affil.walmart.com/t/api02?l=http%3A%2F%2Fwww.walmart.com%2Fip%2FNordicWare-Heavyweight-Scone-Cornbread-Pan%2F7933617%3Faffp1%3DpjiPu5Y7cvNmz4xZOAs5j7QlW2mZPVmc1DR3BvmrkB4%26affilsrc%3Dapi%26veh%3Daff%26wmlspartner%3Dreadonlyapi","customerRating":"4.7","numReviews":20,"customerRatingImage":"http://i2.walmartimages.com/i/CustRating/4_7.gif","categoryNode":"4044_623679_133020","bundle":false,"stock":"Available","addToCartUrl":"http://c.affil.walmart.com/t/api02?l=http%3A%2F%2Faffil.walmart.com%2Fcart%2FaddToCart%3Fitems%3D7933617%7C1%26affp1%3DpjiPu5Y7cvNmz4xZOAs5j7QlW2mZPVmc1DR3BvmrkB4%26affilsrc%3Dapi%26veh%3Daff%26wmlspartner%3Dreadonlyapi","affiliateAddToCartUrl":"http://linksynergy.walmart.com/fs-bin/click?id=|LSNID|&offerid=223073.7200&type=14&catid=8&subid=0&hid=7200&tmpid=1082&RD_PARM1=http%253A%252F%252Faffil.walmart.com%252Fcart%252FaddToCart%253Fitems%253D7933617%257C1%2526affp1%253DpjiPu5Y7cvNmz4xZOAs5j7QlW2mZPVmc1DR3BvmrkB4%2526affilsrc%253Dapi","giftOptions":

Answer 1

What you see in our output is:

print request: open connection to an url - this seems correct and not surprising

print objects: as the output tells, it is an generator and you would probably expect list of values. But as objects are really a generator (you asked for this by using ijson) you shall consume the values from it. Typicall you do it by list(objects)

print products: also a generator, but this time as result of list comprehension. As you used () around the expression, you asked for a generator. If you would use [o for o in objects if o ['type' == 'name']], you would get directly the list. The solution is as with objects: consume the values, e.g. by list(products).

Be aware, that once you consume a value (or all of them) from a generator, they are gone as generator maintains its private internal status, which is changing by each call.

For more see the SO question Convert generator object to list for debugging.