0

I'm trying to make a recursive try-except function in order to deal with occasional errors. In pandas, we can create quantiles for dataframes, however, it might happen that the borders of two or more quantiles coincide and, thus, there isn't a border actually. Sopandas will throw an error. To circumvent this, you can just lower the number of quantiles and that's what I'm attempting to do here.

import pandas as pd

quantiled, dict_bins = recursive_lower_labels(model_quant = model_quant,
                                              n_quantiles = n_quantiles,
                                              reverse = reverse)
def recursive_lower_labels(model_quant,
                           n_quantiles,
                           reverse = False):
    '''
    Recursively lowers the number of labels until it works.
    '''

    if n_quantiles == 0: # base case
        return 'Error: There are no Quantiles to be used.'

    # Not very important... 
    # I'm using this to either use normal or reverse labels.
    if reverse == False:
        labels = range(1, n_quantiles + 1)
    elif reverse == True:
        labels = range(n_quantiles, 0, -1)

    try:

        qt, dc = pd.qcut(model_quant, 
                         q = n_quantiles,
                         labels = labels,
                         retbins = True)

        return qt, dc

    except:

        recursive_lower_labels(model_quant,
                               n_quantiles = n_quantiles - 1,
                               reverse = reverse)

The error I'm getting is (pointing to the function call up top):

cannot unpack non-iterable NoneType object

I suspect it's one of two mistakes I'm making:

  1. There is a problem with scoping somewhere. Maybe n_quantiles? It doesn't seem likely, from my unexperienced debugging.
  2. There is an issue with respect to placing a return before the function recursive call inside the except statement. I've tried a lot of combinations here, even with an extra else at the end and it didn't work either.

By the way, if not recursive, this does work.

EDIT:

My question was marked as a duplicate and this edit is to address that evaluation. Firstly, it was marked as a duplicate of a question that was also marked as such, which is strange, but not that relevant. The important and useful concept that differs those questions from mine is that they both had functions that, although recursive, did not necessarily return something all the time, while mine did return something always and, thus, made it seem that the return on the recursion was not necessary — which turns out to not be true.

Philippe Fanaro
  • 6,148
  • 6
  • 38
  • 76
  • why are you testing for == False or == 0? just use ‘not’ – Alec May 21 '19 at 20:05
  • also for true just say if x – Alec May 21 '19 at 20:05
  • No, the most obvious mistakes are: 1. Incorrect indentation. 2. Putting the function call before the function definition. Other than that your code seems to work with my data. – Stop harming Monica May 21 '19 at 20:23
  • I know you don't need to use `==` when the values are boolean, but it's just easier to read this way. The incorrect indentation is just a minor slip up when editing the post. And having the function call before the function is not actually what I did in my code, it just seemed nicer to read here. – Philippe Fanaro May 21 '19 at 20:29
  • 2
    It's nicer to have code that actually runs and reproduces the failure. – Mark Tolonen May 21 '19 at 20:38
  • I didn't put something that runs here because the data I was feeding it to see if it works is kind of confidential and I haven't yet figured out how to create an artificial dataset that emulates the issues I was having in this particular instance. – Philippe Fanaro May 21 '19 at 22:51

1 Answers1

1

All you have to do is return your recursion. With some light refactoring:

def recursive_lower_labels(model_quant,
                           n_quantiles,
                           reverse=False):
    """
     Recursively lowers the number of labels until it works.
    """
    if n_quantiles == 0:  # base case
        return 'Error: There are no Quantiles to be used.'

    # Not very important...
    # I'm using this to either use normal or reverse labels.
    if reverse:
        labels = range(n_quantiles, 0, -1)
    else:
        labels = range(1, n_quantiles + 1)

    try:
        return pd.qcut(model_quant,
                         q=n_quantiles,
                         labels=labels,
                         retbins=True)

    except:
        return recursive_lower_labels(model_quant,
                                      n_quantiles=n_quantiles - 1,
                                      reverse=reverse)
bsplosion
  • 2,641
  • 27
  • 38
  • I had tried this at some point but it wasn't working, I'm gonna try it again tomorrow and see if it works. – Philippe Fanaro May 21 '19 at 22:48
  • But I wonder why you would need to use `return` here, isn't the `return` inside the `try` already enough? – Philippe Fanaro May 21 '19 at 22:53
  • Despite not quite understanding the need for a `return`, it did the job. Thank you very much. – Philippe Fanaro May 22 '19 at 10:58
  • @PhilippeFanaro, the return is required due to your usage of recursion - otherwise, the controlling function call actually wouldn't return anything, despite the child layer having a return value. Following the duplicate question's recursion (ha), [check out roippi's answer](https://stackoverflow.com/a/17778390/2738164) for a nice explanation of why a return is necessary. – bsplosion May 22 '19 at 12:39
  • After thinking for a while I think it's way simpler to look at it as a *scoping problem* --- rather than a new concept like "falling off the end of a function" --- : if I don't put the `return`, the recursion will return the value but only inside the main function, while the outside function call will have nothing to return. – Philippe Fanaro May 22 '19 at 13:58