0

I am dealing with an existing test suite, where we want to implement a timeout functionality, which will cause a hanging test to time out and then move on with its regular teardown/cleanup.

I am toying with the idea of running each test in a process, which I can terminate after e.g. a timeout of 3 seconds. Ideally, I don't want to modify the test cases and instead just add a decorator indicating the test is affected by this timeout behavior.

This is what I have, a minimal example:

import multiprocessing
import sys
from time import sleep


def timeout(func):
    def wrapper():
        proc = multiprocessing.Process(target=func)
        proc.start()
        sleep(3)
        proc.terminate()
    return wrapper


@timeout
def my_test():
    while True:
        sleep(1)


if __name__ == "__main__":
    my_test()

But for some reason, it seems pickle cannot deal with this and the decorator somehow messes up the reference to the function, as this error is hit:

$ python multiproc.py
2019-11-07 07:34:37.098 | DEBUG    | __main__:wrapper:13 - In wrapper
Traceback (most recent call last):
  File "multiproc.py", line 30, in <module>
    my_test()
  File "multiproc.py", line 15, in wrapper
    proc.start()
  File "C:\Python36\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Python36\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Python36\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Python36\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Python36\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function my_test at 0x000001C0A7E87400>: it's not the same object as __main__.my_test
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python36\lib\multiprocessing\spawn.py", line 99, in spawn_main
    new_handle = reduction.steal_handle(parent_pid, pipe_handle)
  File "C:\Python36\lib\multiprocessing\reduction.py", line 82, in steal_handle
    _winapi.PROCESS_DUP_HANDLE, False, source_pid)
OSError: [WinError 87] The parameter is incorrect

Does anyone have an idea if this can be solved without modifying the existing test case?

fredrik
  • 9,631
  • 16
  • 72
  • 132
  • okay, I just removed the logging altogether – fredrik Nov 07 '19 at 08:00
  • The very first link I get via google search explaines the origin of the problem https://stackoverflow.com/questions/1412787/picklingerror-cant-pickle-class-decimal-decimal-its-not-the-same-object. Possible solution is `timeout(my_test())` instead of decoration. –  Nov 07 '19 at 09:54
  • FYI... Running the code Ubuntu and Python 3.6, doesn't produce the issue. – bivouac0 Nov 07 '19 at 14:17

0 Answers0