Why am I seeing ZombieProcess exceptions when running Python tests with nox on MacOS?

Question

I'm running Python tests with nox for the first time, and although the tests appear to be passing I'm consistently seeing the following (this has nothing to do with boto3, it's happening on all my tests with a wide variety of dependencies):

======================================================================== 1 passed in 0.01s ========================================================================
nox > Session my_boto3_test(boto3_version='1.22.13') raised exception psutil.ZombieProcess(pid=86804, msg="PID still exists but it's a zombie")
Traceback (most recent call last):
  File "/project_path/venv/lib/python3.9/site-packages/psutil/_psosx.py", line 343, in wrapper
    return fun(self, *args, **kwargs)
  File "/project_path/venv/lib/python3.9/site-packages/psutil/_psosx.py", line 401, in cmdline
    return cext.proc_cmdline(self.pid)
ProcessLookupError: [Errno 3] assume no such process (originated from sysctl(KERN_PROCARGS2) -> EINVAL)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project_path/venv/lib/python3.9/site-packages/nox/sessions.py", line 695, in execute
    self.func(session)
  File "/project_path/venv/lib/python3.9/site-packages/nox/_decorators.py", line 113, in __call__
    return super().__call__(*args, **kwargs)
  File "/project_path/venv/lib/python3.9/site-packages/nox/_decorators.py", line 68, in __call__
    return self.func(*args, **kwargs)
  File "/project_path/venv/lib/python3.9/site-packages/nox/_decorators.py", line 68, in __call__
    return self.func(*args, **kwargs)
  File "/project_path/noxfile.py", line 191, in integration_tests_boto3_sqs
    kill_process_and_clean_outputs(full_path, "run_app", session)
  File "/project_path/noxfile.py", line 647, in kill_process_and_clean_outputs
    kill_process(process_name)
  File "/project_path/noxfile.py", line 657, in kill_process
    if proc.name() == process_name:
  File "/project_path/venv/lib/python3.9/site-packages/psutil/__init__.py", line 628, in name
    cmdline = self.cmdline()
  File "/project_path/venv/lib/python3.9/site-packages/psutil/__init__.py", line 681, in cmdline
    return self._proc.cmdline()
  File "/project_path/venv/lib/python3.9/site-packages/psutil/_psosx.py", line 346, in wrapper
    raise ZombieProcess(self.pid, self._name, self._ppid)
psutil.ZombieProcess: PID still exists but it's a zombie (pid=86804)
nox > Session my_boto3_test(boto3_version='1.22.13') failed.

I've searched high and low, but I can't find anything documenting this nor a strategy to deal with this, and aside from messing with my OCD I'm not convinced it's even a real problem.

The tests are configured with noxfile.py and are running perfectly via GitHub Actions (no warnings or errors, certainly no mentions of any ZombieProcesses. I'm running it locally from within a virtualenv, with the command:

python3 -m nox

(Same command as the GitHub Actions config uses, all dependencies installed using the same requirements.txt and setup.py)

score 0 · Answer 1 · answered Jun 27 '23 at 10:31

This turned out to have nothing to do with nox itself, it's a specific call to kill processes using the psutil module that I hadn't noticed (yay, other people's code) that doesn't run nicely on Darwin.

The solution (for anyone interested) was wrapping the proc.kill() statements in try / except clauses.

Why am I seeing ZombieProcess exceptions when running Python tests with nox on MacOS?

1 Answers1