1

I am writing code to search the New York Times API for news articles and to print information from those articles.

I have been able to get the URLs for the news articles, but when I try to read the information from them, I am getting the following error:

urllib.error.HTTPError: HTTP Error 303: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
See Other

My code is pasted below and I commented the line where I am getting the error. The thing that confuses me is that urlopen worked fine the first time I used it to read information from NYT's API, but resulted in an error the second time when I try to read information from the URL of the specific article.

from urllib.request import urlopen
import json

url = 'http://api.nytimes.com/svc/search/v2/articlesearch.json?q=olympics&api-key=mykey'
request = urlopen(url) #no problems here

request = request.readall().decode('utf-8') 

json_obj = json.loads(request)


for i, item in enumerate(json_obj):
    title = json_obj.get('response').get('docs')[i].get('headline').get('main')
    web_url = json_obj.get('response').get('docs')[i].get('web_url')
    print(i+1)
    print(title)
    print(web_url)
    print()
    request = urlopen(web_url) #This is where I am getting the error
    request = request.read().decode('utf-8') 
    print (request)

I am using portable python 3.2.5.1

I am also new to using python. Most of my programming experience is with java.

Deena
  • 343
  • 1
  • 11
  • http://stackoverflow.com/questions/9926023/handling-rss-redirects-with-python-urllib2 – Guy Gavriely Feb 13 '14 at 03:29
  • @GuyGavriely Would this solution work in python 3? The import statement `from cookielib import CookieJar` is not working for me. I searched google to find a python 3 equivalent, but haven't been able to find one. – Deena Feb 13 '14 at 03:42
  • 2
    `from http.cookiejar import CookieJar` see http://docs.python.org/3.0/library/http.cookiejar.html#http.cookiejar.CookieJar – Guy Gavriely Feb 13 '14 at 03:51

0 Answers0