1
def store(self) -> list:
    result = []
    for url in self.urls():
        if url.should_store():
            stored_url = self.func_that_can_throw_errors(url)
            if stored_url: result.append(stored_url)
    return result

Preface: not actual method names. Silly names chosen to emphasize

During the loop errors may occur. In that case, I desire the intermediate result to be returned by store() and still raise the original exception for handling in a different place.

Doing something like

try:
    <accumulating results ... might break>
except Exception:
    return result
    raise

sadly doesn't do the trick, since trivially the raise stmt won't be reached (and thus an empty list get's returned).

Do you guys have recommendations on how not to lose the intermediate result?

Thanks a lot in advance - Cheers!

Community
  • 1
  • 1
DrumTim
  • 101
  • 3
  • 11
  • That is not possible as far as I know, you cannot both throw an exception and return a value, control-flow-wise you have to do one or the other. You could maybe do something like maybe returning a tagged union - one case is an exception value, and the other is the actual value (and maybe a third case with both the "result-so-far" as well as the exception). Or make a custom exception that also contains the "result-so-far". – MelvinWM Jun 04 '20 at 08:37
  • In Python, you might instead of a tagged union return a tuple containing the exception and the "result-so-far". Though any callers of that function should then take care to check whether what the form of the return value is. I don't know what is idiomatic in Python reg. that. – MelvinWM Jun 04 '20 at 08:39
  • 1
    The best approach in Python might be to always return a tuple, let the first part always be the (intermediate/complete) result (if any) and the second part always be an exception thrown (if any). That does require callers of the function to always have to be careful to check the various possibilities reg. the return value of the function, such as only a result, only an exception, and both a result and an exception. – MelvinWM Jun 04 '20 at 08:44
  • 1
    Ah, reminds me of Go, where it appeared to be pretty common to have result, err = func() statements. Downside in this particular case to me is that I'd need to adjust every call of store() and, since I'd like the original exception to be thrown anyway, the calling statement would somehow have to make sure that any given exception is actually raised after processing the (intermediate) result. Ah well, good to have had this conversation. I sense there are other issues comming up... thanks! :)) – DrumTim Jun 04 '20 at 09:01
  • 1
    Yup... to have an use of the intermediate result I'd have to surround every .store() statement with try-except anyway in order to process the result from it. – DrumTim Jun 04 '20 at 09:05

3 Answers3

2

It is not possible as you imagine it. You can't raise an exception and return a value.

So I think what you are asking for is a work around. There, I see two possibilities:

  1. return a Flag/Exception along the actual return value:

Return flag: 

except Exception:
    return result, False

where False is the Flag telling that something went wrong

Return Exception:

except Exception as e:
    return result, e
  1. Since it appears, that store is a method of some class, you could raise the exception and retrieve the intermediary result with a second call like so:
def store(self):
    result = []
    try:
        # something
    except Exception:
        self.intermediary_result = result
        raise

def retrieve_intermediary(self):
    return self.intermediary_result
attalos
  • 454
  • 5
  • 9
2

The best answer I can come up with given my limited knowledge of Python would be to always return a pair, where the first part of the pair is the result and the second part of the pair is an optional exception value.

def store(self) -> list:
    '''
    TODO: Insert documentation here.

    If an error occurs during the operation, a partial list of results along with
    the exception value will be returned.

    :return A tuple of [list of results, exception]. The exception part may be None.
    '''

    result = []

    for url in self.urls():

        if url.should_store():
            try:
                stored_url = self.func_that_can_throw_errors(url)
            except Exception as e:
                return result, e

            if stored_url: result.append(stored_url)

    return result, None

That said, as you have mentioned, if you have this call multiple places in your code, you would have to be careful to change it in all relevant places as well as possibly change the handling. Type checking might be helpful there, though I only have very limited knowledge of Python's type hints.

MelvinWM
  • 749
  • 4
  • 13
  • 1
    Looks reasonable. In regards of exception handling, if you return the Exception e, you might want to make sure to keep the stack trace when raising e. See https://stackoverflow.com/questions/8760267/re-raise-python-exception-and-preserve-stack-trace In combination with that, the tuple solution would be a good one as well. – DrumTim Jun 04 '20 at 09:16
1

Meanwhile I had the idea to just use an accumulator which appears to be the 'quickest' fix for now with the least amount of changes in the project where store() is called. The (intermediate) result is not needed everywhere (let's say it's optional). So...

I'd like to share that with you:

    def store(self, result_accu=None) -> list:
        if result_accu is None:
            result_accu = []
        for url in self.urls():
            if url.should_store():
                stored_url = self.func(url)
                if stored_url: result_accu.append(stored_url)
        return result_accu

Still returning a list but alongside the intermediate result is accessible by reference on the accu list. Making the parameter optional enables to leave most statements in project as they are since the result is not needed everywhere.

store() is rather some kind of a command where the most work on data integrity is done within already. The result is nice-to-have for now.

But you guys also enabled me to notice that there's work to do in ordner to process the intermediate result anyway. Thanks! @attalos @MelvinWM

DrumTim
  • 101
  • 3
  • 11