Passing method call as a variable to error handling function

Question

Trying to implement simple error handling without adding buku try / except statements to my code.

My if_error function tries to emulate the iferror(value,value_if_error) formula in excel.

If the value (another formula) is valid, 
     return its resulting value 
else 
     return value_if_error

How can I pass a method call from a beautifulsoup object (soup) with parameters to a generic try/ except function?

Tried lambda but didn't understand enough to make it work with parameters & soup.
Looked at Partial but didn't see how that would call the beautiful soup method
Looked at this but didn't see how soup would be passed?

My Code:

def if_error(fn,fail_value):
    try:
      value = fn
    except:
      value = fail_value
    return value

def get_trulia_data(soup):
    d = dict()

    description = if_error(soup.find('div', attrs={'class': 'listing_description_module description'}).text,'')

    sale_price = if_error(soup.find('div', attrs={'class': 'price'}).text,'0')
    sale_price = re.sub('[^0-9]', '', sale_price)

    details = if_error(soup.find('ul', attrs={'class': 'listing_info clearfix'}),'')

    bed = if_error(soup.find('input', attrs={'id': 'property_detail_beds_org'})['value'],'')
    bath = if_error(soup.find('input', attrs={'id': 'property_detail_baths_org'})['value'],'')

    ...

    return d

Error:

Traceback (most recent call last):
data_dict = get_trulia_data(url)
description = if_error(soup.find('div', attrs={'class': 'listing_description_module description'}).text,'')
AttributeError: 'NoneType' object has no attribute 'text'

The soup.find method keeps firing before the if_error function is reached. How can I fix this?

You may already know this, but using a bare `except` is dangerous practice as it will catch just about every possible problem. You should supply the Exceptions you are willing to have converted to the `if_error` value. — Ethan Furman, Jan 19 '12 at 23:08

score 1 · Accepted Answer · answered Jan 19 '12 at 21:07

1

How about this:

def if_error(fn, fail_value, *args, **kwargs):
    try:
      return fn(*args, **kwargs)
    except:
        return fail_value

def test_fail(x):
    raise ValueError(x)

def test_pass(x):
    return x

if __name__=='__main__':
    print if_error(test_fail, 0, 4)
    print if_error(test_pass, 0, 5)

answered Jan 19 '12 at 21:07

Meitham

9,178
5
34
45

solution works. slightly more readable than building an argument dict `description = if_error(soup.find,'','div',attrs={'class':'listing_description_module description'})` – mitrebox Jan 19 '12 at 22:37

score 0 · Answer 2 · answered Jan 19 '12 at 20:56

0

Your problem is that you are passing the result of soup.find() to if_error instead of the function. It's possible that you could try passing the actual function to if_error but I would do this instead:

def findError(soup, arg1, arg2, error):
    try:
        return soup.find(arg1, arg2)
    except:
        return error

and then call:

findError(soup, 'div', attrs={}, '')

answered Jan 19 '12 at 20:56

thagorn

727
7
14

non-keyword arg after arg with the code `description = findError(soup,'div', attrs={'class':'listing_description_module description'},'')` /changed to `description = findError(soup,'div', {'class':'listing_description_module description'},'')` /and findError function to `return soup.find(arg1, attrs=arg2)` – mitrebox Jan 19 '12 at 21:42
Passing functions is not only possible in Python, it is encouraged. Check out the answer by Meitham for a good way to handle this. – Ethan Furman Jan 19 '12 at 23:05
What is the point of `attrs={}`? You have `findError` accepting `soup`, `arg1`, `arg2`, and `error`, bot not an `attrs`. – Ethan Furman Jan 19 '12 at 23:06
Meitham's solution would be the what you wanted for sure – thagorn Jan 19 '12 at 23:07
@thagorn agreed. Meitham's solution answered the issue in the way it was asked, passing 'a' fn. However your's was useful in illustrating a solution tailored to beautifulsoup in a simple, understandable manner. – mitrebox Jan 20 '12 at 05:27

score 0 · Answer 3 · answered Jan 19 '12 at 20:57

0

What about doing something like this?

def if_error(f,f_args,fail_value):
    try:
      value = f(**f_args).text
    except:
      value = fail_value
    return value

where f_args is a dictionary of the arguments you want to pass to your function f.

answered Jan 19 '12 at 20:57

aganders3

5,838
26
30

Sweet! I finally understand that. This does effectively let name a function in one place and execute it in another. Good re-usability. `arg_dict = {'name':'div','attrs':{'class':'listing_description_module description'}} description = if_error(soup.find,arg_dict,'')` – mitrebox Jan 19 '12 at 22:15

score 0 · Answer 4 · answered Jan 19 '12 at 23:20

You'll want to supply several items to your error-masking function: the exceptions you want to convert to a default value, the default value, the function to call, and its arguments:

def if_error(exceptions, fail_value, fn, *args, **kwargs):
    try:
      return fn(*args, **kwargs)
    except exceptions:
        return fail_value

def test_fn(x):
    return int(x)

if __name__=='__main__':
    print if_error(ValueError, 0, test_fn, '42')
    print if_error(ValueError, -1, test_fn, 'abc')
    print if_error(TypeError, -2, test_fn, 'abc')

Which gives us:

42
-1
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    print if_error(TypeError, -2, test_fn, 'abc')
  File "test.py", line 3, in if_error
    return fn(*args, **kwargs)
  File "test.py", line 8, in test_fn
    return int(x)
ValueError: invalid literal for int() with base 10: 'abc'

As you can see, the last call let the exception raise because we weren't catching ValueErrors with it.

Passing method call as a variable to error handling function

4 Answers4