using try and except for searching urls using google api in python

Question

I want to search urls in a list, which add up to approximately 74 urls. I used try and except to tell python skip sites that do not respond (with http error code 400, etc.)

from googleapiclient.discovery import build
service = build("customsearch", "v1", developerKey='keyhere')

for i in range(0,k-1):
    try:
        queries = search_sources[i]
        res = service.cse().list(q= queries, cx='idhere',).execute()
    except urllib2.HTTPError:
        continue

but the problem is that I get the message about some sites has error (which means that except part is not working):

   Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/googleapiclient/http.py", line 838, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/customsearch/v1?q=site%3Ahttp%3A%2F%2Fbit.ly%2FgktvnmChina+protest&alt=json&cx=myid&key=mykey returned "Bad Request">

it says googleapiclient.errors.HttpError so use that in except — Shivkumar kondi, Feb 03 '17 at 05:12
You should probably just do `for queries in search_sources:` instead of manually manipulating `i` as an index. — skrrgwasme, Feb 03 '17 at 05:14

score 2 · Answer 1 · answered Feb 03 '17 at 05:11

2

You need to catch the correct HttpError. Read the trace.

googleapiclient.errors.HttpError:

So import that

from googleapiclient.errors import HttpError as GoogleHttpError

then

except GoogleHttpError:

answered Feb 03 '17 at 05:11

OneCricketeer

179,855
19
132
245

Thanks. The code worked, but I'm not getting anything for res? – halo09876 Feb 03 '17 at 05:15
I'm just answering the asked questions question. Don't know how that library works, but it also looks like you aren't using or checking the res value within your loop – OneCricketeer Feb 03 '17 at 05:17

Son of a Beach · Answer 2 · 2017-02-03T05:30:12.113

0

According to the raise line in your exception output, I think the exception handler you want would be:

except HttpError:
    continue

EDIT: You'll need to import this exception, similar to @cricket_007 answer.

Or just use the generic:

except Exception:
    continue

edited Feb 03 '17 at 05:30

answered Feb 03 '17 at 05:07

Son of a Beach

1,733
1
11
29

I get "HttpError" not defined – halo09876 Feb 03 '17 at 05:08
For that one, the code worked but I'm not getting any results for res – halo09876 Feb 03 '17 at 05:14
Do you mean you don't get a `res` value when there is an exception, or when there is not an exception? You certainly should not get a `res` when there is an exception. – Son of a Beach Feb 03 '17 at 05:16
yes it's strange. I do res and I get message >>> res Traceback (most recent call last): File "", line 1, in NameError: name 'res' is not defined – halo09876 Feb 03 '17 at 05:17
You need to post your code of where you're using `res`. Is it out of scope (and therefore not defined)? Also, see: http://stackoverflow.com/questions/25666853/how-to-make-a-variable-inside-a-try-except-block-public – Son of a Beach Feb 03 '17 at 05:18
i defined res below try: – halo09876 Feb 03 '17 at 05:24
But where do you use it? Is it in the same scope as where you defined it? Also, if you got an exception on the line you tried to define it, then it is not defined. – Son of a Beach Feb 03 '17 at 05:25

using try and except for searching urls using google api in python

2 Answers2