how do I catch urllib.error.HTTPError: HTTP Error 403: Forbidden?

Question

I'm writing a Flask app where I call a method that might return 403. I figured the following code would handle the error appropriately:

try:
    connection = myLib.Login(username, password)
except urllib.error.HTTPError as err:
    abort(err.code)

But that does not seem to work. In case myLib.Login returns 403 I get the following:

[2019-05-22 15:22:15,798] ERROR in app: Exception on /api/users [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "./app/routes.py", line 16, in authenticate
    connection = myLib.Login(username, password)

  File "/usr/local/lib/python3.7/site-packages/mylib.py", line 90, in __open
    resp = opener.open(req)
  File "/usr/local/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

How can I catch this error and return 403?

In a regular Python script the exception triggers just fine so I suspect this might be related to flask and/or uwsgi somehow.

Possible duplicate of [urllib2.HTTPError: HTTP Error 403: Forbidden](https://stackoverflow.com/questions/13303449/urllib2-httperror-http-error-403-forbidden) — jose_bacoy, May 22 '19 at 13:42
Exception doesnt throw. However, if I write a simple Python script (without flask or uwsgi obviously) the exception triggers just fine. Any ideas? — ardevd, May 22 '19 at 18:23

Randolph Carter · Answer 1 · 2019-05-22T14:11:15.080

This is most likely an issue of

connection = myLib.Login(username, password)

being asynchronous. myLib.Login probably spawns a child thread. Parents can not catch exceptions raised in their children as they have their own context. Refer to Catch a thread's exception in the caller thread in Python for more info.

Edit:

If you want to wait ("block") until the connection has been established and then decide whether to continue or abort, you can use queues to establish communication between threads:

import queue
notifierQueue = queue.Queue()
connection = myLib.Login(username, password, notifierQueue)

response = notifierQueue.get(block=True)
if response [some condition]: 
    abort(err.code)
else:
    do whatever

Queue.get() with blocking set to true (default) will wait until an item is available in the queue. Handling the HTTPError needs to happen in the Login part and then put(). You can also set a timeout parameter for the get() which will result in an exception being raised if there is no element in the queue after the time is up.

I don't think that's the issue but I'll check and see. Considering the fact that the flask function returns 200 OK if the connection is successful and 500 if the 403 error occurs I'm assuming it's not entirely asynchronous. — ardevd, May 22 '19 at 16:16
Can it be related to uwsgi and/or flask? If I write a simple python script the exception triggers just fine — ardevd, May 22 '19 at 18:22

score 0 · Answer 2 · answered May 22 '19 at 19:04

0

The exception was working as intended. My devops chain as the issue.

answered May 22 '19 at 19:04

ardevd

3,329
5
30
55

how do I catch urllib.error.HTTPError: HTTP Error 403: Forbidden?

2 Answers2