0

I have following code that do some work in ProcessPoolExecutor and raise the exception:

import asyncio
import concurrent.futures
import platform


class MissingValue(Exception):
    def __init__(self, value_name: str) -> None:
        super().__init__(f"Missing value: '{value_name}'")


def _do_work() -> None:
    raise MissingValue("val1")


async def _main() -> None:
    with concurrent.futures.ProcessPoolExecutor(max_workers=2) as proc_pool:
        try:
            await asyncio.get_running_loop().run_in_executor(proc_pool, _do_work)
        except Exception as e:
            print(str(e))


if __name__ == "__main__":
    if platform.system() == "Windows":
        asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

    asyncio.run(_main())

I expect that it will output:

Missing value: 'val1'

but it output:

Missing value: 'Missing value: 'val1''

So, MissingValue wrapping twice.

Why it happens? Is it bug in asyncio?

dzav
  • 545
  • 1
  • 10
  • 25
  • 1
    This I'm not 100% sure off, but I'm pretty sure it has something to do with how Windows spawns new instances to be executed as a system process. If you switch to using `ThreadPoolExecutor` instead, it works as intended. Same goes for Linux. – Cow Aug 12 '22 at 12:00
  • you might want refer this https://stackoverflow.com/a/18108623/7887883 – Pavan Kumar T S Aug 12 '22 at 12:02
  • @PavanKumarTS Looks like you are correct. – Cow Aug 12 '22 at 12:06
  • If I replace `concurrent.futures.ProcessPoolExecutor` to `concurrent.futures.ThreadPoolExecutor` responce is correct: `Missing value: 'val1'`. – dzav Aug 12 '22 at 12:07
  • I checked `concurrent.futures.ProcessPoolExecutor` in Linux: same duplicate output: `Missing value: 'Missing value: 'val1''` – dzav Aug 12 '22 at 13:12

1 Answers1

0

If you change your class as per the link from Pavan Kumar:

class MissingValue(Exception):
    def __init__(self, value_name: str) -> None:
        self.message = value_name
    def __str__(self):
        return(f"Missing value: '{self.message}'")

The error is being returned correctly using ProcessPoolExecutor:

Missing value: 'val1'
Cow
  • 2,543
  • 4
  • 13
  • 25
  • It will work, but it is not a solution in general: it is a hack and sometimes I catch exceptions that was created not in my code (in a library for example). – dzav Aug 12 '22 at 13:15