1

This works fine in development environment:

url = "http://www.google.com/"
return urllib2.urlopen(url)

But when I upload it to google apps engine and run it there, I got the following error:

    return urllib2.urlopen(url)
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 127, in urlopen
    return _opener.open(url, data, timeout)
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 1214, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/urllib2.py", line 1184, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno 13] Permission denied>

Anyone know why this happening? thanks so much!

Kelly L
  • 31
  • 1
  • 2
  • possible duplicate: http://stackoverflow.com/questions/28081350/what-causes-urlopen-error-errno-13-permission-denied-errors – jvdh Oct 06 '15 at 11:32
  • @jvdh GAE uses urllib2 only as wrapper of its own implementation (the requests are handled by GCP resources), so I don't think the questions are related. – Ani Oct 06 '15 at 13:02
  • 1
    Have you tried the same code with a different URL? Best would be something non-Google, non-redirecting, and non-SSL. For example: `url = "http://www.heise.de"` – Ani Oct 06 '15 at 13:15
  • @Ani: Thanks so much for the help. You are absolutely right. "http://www.heise.de" works fine but not google.com. Actually i'm trying to use urllib2.urlopen on this: "https://www.google.com/recaptcha/api/siteverify" and that's causing permission denied error. Anyway that I can get around it? Thanks so much!! – Kelly L Oct 08 '15 at 00:25
  • @KellyL: I think fetching from Google services requires you some additional thing. In the docs is a section about this, heading: "Making requests to another App Engine app or Google service" https://cloud.google.com/appengine/docs/python/urlfetch/ – Ani Oct 08 '15 at 21:54
  • @Ani Awesome. Thanks so much! will give it a try :) – Kelly L Oct 09 '15 at 06:11

1 Answers1

0

This is probably a bot or DDoS protection feature Google has implemented. Google probably does not want people to host services on their own GAE that in turn load their home page. Once is innocent perhaps, but imagine putting your urlopen in an infinite loop. Google's own infrastructure could then be used to attack it's own home page. Explains why (most) other URLs work.

Some URLs (ex: Amazon product pages) will 403 when submitted via a urlopen request from Google Cloud-associated IPs. This is implemented by Amazon to prevent bots, content scraping, DoS, etc. on Amazon. Same principle.

B Lowery
  • 21
  • 3