5

I'm trying to run bunch of linux commands on multiple devices parallely using python multiprocessing module. script is able to connect to multiple devices serially and starts executing the linux commands parallely. the script fails in the first attempt but it is successful from the second attempt.

Please find the error log:

    -E- !!! *** Exception occurred during execution *** !!!
    -E- Exception Name: ExceptionPexpect
    -E- Exception trace back: 
        Traceback (most recent call last):
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/ptyprocess/ptyprocess.py", line 705, in isalive
        pid, status = os.waitpid(self.pid, waitpid_options)
    ChildProcessError: [Errno 10] No child processes

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/pexpect/pty_spawn.py", line 23, in _wrap_ptyprocess_err
        yield
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/pexpect/pty_spawn.py", line 705, in isalive
        alive = ptyproc.isalive()
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/ptyprocess/ptyprocess.py", line 712, in isalive
        'on our process?')
    ptyprocess.util.PtyProcessError: isalive() encountered condition where "terminated" is 0, but there was no child process. Did someone else call waitpid() on our process?

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/warriorframework_py3/warrior/Framework/ClassUtils/WNetwork/warrior_cli_class.py", line 1627, in send_command
        if end_prompt else -1))
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/pexpect/spawnbase.py", line 344, in expect
        timeout, searchwindowsize, async_)
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/pexpect/spawnbase.py", line 372, in expect_list
        return exp.expect_loop(timeout)
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/pexpect/expect.py", line 169, in expect_loop
        incoming = spawn.read_nonblocking(spawn.maxread, timeout)
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/pexpect/pty_spawn.py", line 478, in read_nonblocking
        if not self.isalive():
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/pexpect/pty_spawn.py", line 705, in isalive
        alive = ptyproc.isalive()
      File "/usr/lib/python3.5/contextlib.py", line 77, in __exit__
        self.gen.throw(type, value, traceback)
      File "/home/venkat/Desktop/setup_cleanup_with_parallel/test/lib/python3.5/site-packages/pexpect/pty_spawn.py", line 25, in _wrap_ptyprocess_err
        raise ExceptionPexpect(*e.args)
    pexpect.exceptions.ExceptionPexpect: isalive() encountered condition where "terminated" is 0, but there was no child process. Did someone else call waitpid() on our process?

code:

    target_module = class to do the operation
    tc_args_dict = args
    jobs_list = []
    jobs_list = None

    for target_module in case_list:
        process, jobs_list, output_q = create_and_start_process_with_queue(target_module,
                                                                       tc_args_dict,
                                                                       jobs_list, output_q)

    print_debug("process: {0}".format(process))
    for job in jobs_list:
        job.join()

    def create_and_start_process_with_queue(target_module, tc_args_dict, jobs_list, output_q, p_name='')
        if output_q is None:
            utput_q = multiprocessing.Manager().Queue()

        args_dict["output_q"] = output_q
        args_list = []
        for _, value in args_dict.items():
            args_list.append(value)
        args_tuple = tuple(args_list)
        process = multiprocessing.Process(name=p_name, target=target_module, args=args_tuple)
        jobs_list.append(process)
        process.start()

        return process, jobs_list, output_q

Can any one help here please.

Venkat
  • 51
  • 3
  • I'd put my money on `target module` instantiating a pexpect instance in the main process. That gets passed to the first process which tears it down when it's done. Second process was handed the same instance which is now in a completely different state. Need to see `target module` to confirm. – Guy Gangemi Feb 17 '23 at 03:54

0 Answers0