Python pytest hangs. For instance, "pytest --version" simply hangs

Question

The following hangs:

PS C:\Users\Fowler> pytest --version

Notes:

I am in Windows 10.
By hang, I mean at least 5 minutes of waiting for the pytest --version to return...
While waiting for pytest, python.exe is using 100% of a logical processor on my computer.
I uninstalled all python installations with windows installer and I reinistalled python 3.8.0 in an attempt to fix.
pytest only fails when I am not using a venv. So, pytest does work using a venv.
However, I can't use a venv with vscode, because debugging with venv gives a strange "Session-1 timed out waiting for debuggee to spawn" <-- you would think the word debuggee would be a nice clue, but not much found with that word on google. I am guessing this is a different problem, but maybe related?
In summary, I can't debug python with a venv, and I can't run pytest unit tests without a venv. Probably, these items are unrelated... But, because of this catch-22, I will be sooo grateful for any hints to fix either problem.

When I hit <ctrl-c> to break out of the pytest "hang", the following is displayed (but changes a little bit at the end each time?:

Traceback (most recent call last):
  File "c:\program files\python38\lib\runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\program files\python38\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\Scripts\pytest.exe\__main__.py", line 7, in <module>
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\config\__init__.py", line 72, in main
    config = _prepareconfig(args, plugins)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\config\__init__.py", line 222, in _prepareconfig
    return pluginmanager.hook.pytest_cmdline_parse(
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\pluggy\hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\pluggy\manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\pluggy\manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\pluggy\callers.py", line 203, in _multicall
    gen.send(outcome)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\config\__init__.py", line 742, in pytest_cmdline_parse
    self.parse(args)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\config\__init__.py", line 948, in parse
    self._preparse(args, addopts=addopts)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\config\__init__.py", line 896, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\pluggy\manager.py", line 299, in load_setuptools_entrypoints
    plugin = ep.load()
  File "c:\program files\python38\lib\importlib\metadata.py", line 75, in load
    module = import_module(match.group('module'))
  File "c:\program files\python38\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\assertion\rewrite.py", line 138, in exec_module
    _write_pyc(state, co, source_stat, pyc)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\assertion\rewrite.py", line 274, in _write_pyc
    with atomic_write(fspath(pyc), mode="wb", overwrite=True) as fp:
  File "c:\program files\python38\lib\contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\atomicwrites\__init__.py", line 156, in _open
    with get_fileobject(**self._open_kwargs) as f:
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\atomicwrites\__init__.py", line 173, in get_fileobject
    descriptor, name = tempfile.mkstemp(suffix=suffix, prefix=prefix,
  File "c:\program files\python38\lib\tempfile.py", line 332, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
  File "c:\program files\python38\lib\tempfile.py", line 247, in _mkstemp_inner
    file = _os.path.join(dir, pre + name + suf)
KeyboardInterrupt

The next time try to run pytest --version and I hit <ctrl-c> it ends with:

Traceback (most recent call last):
  File "c:\program files\python38\lib\runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
...
...
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\atomicwrites\__init__.py", line 173, in get_fileobject
    descriptor, name = tempfile.mkstemp(suffix=suffix, prefix=prefix,
  File "c:\program files\python38\lib\tempfile.py", line 332, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
  File "c:\program files\python38\lib\tempfile.py", line 248, in _mkstemp_inner
    _sys.audit("tempfile.mkstemp", file)
KeyboardInterrupt

The next time try to run pytest --version and I hit <ctrl-c> it ends with:

Traceback (most recent call last):
  File "c:\program files\python38\lib\runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
...
...
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\atomicwrites\__init__.py", line 173, in get_fileobject
    descriptor, name = tempfile.mkstemp(suffix=suffix, prefix=prefix,
  File "c:\program files\python38\lib\tempfile.py", line 332, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
  File "c:\program files\python38\lib\tempfile.py", line 256, in _mkstemp_inner
    if (_os.name == 'nt' and _os.path.isdir(dir) and
  File "c:\program files\python38\lib\genericpath.py", line 42, in isdir
    st = os.stat(s)
KeyboardInterrupt

The next time try to run pytest --version and I hit <ctrl-c> it ends with:

Traceback (most recent call last):
  File "c:\program files\python38\lib\runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
...
...
  File "C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\atomicwrites\__init__.py", line 173, in get_fileobject
    descriptor, name = tempfile.mkstemp(suffix=suffix, prefix=prefix,
  File "c:\program files\python38\lib\tempfile.py", line 332, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
  File "c:\program files\python38\lib\tempfile.py", line 250, in _mkstemp_inner
    fd = _os.open(file, flags, 0o600)
KeyboardInterrupt

I don't know if this output will help, but I thought it might be useful to see all the locations on my machine where python and/or pytest are installed:

PS C:\Users\Fowler> where.exe /r c:\ python
c:\Program Files\Amazon\AWSCLI\runtime\python.exe
c:\Program Files\Amazon\AWSSAMCLI\runtime\python.exe
c:\Program Files\MySQL\MySQL Workbench 8.0 CE\python.exe
c:\Program Files\Python38\python.exe
c:\Program Files\Python38\Lib\venv\scripts\nt\python.exe
c:\Users\Fowler\.vscode\extensions\lextudio.restructuredtext-116.0.0\out\python.js
c:\Users\Fowler\.vscode\extensions\teabyii.ayu-0.18.0\test\Python.py
c:\Users\Fowler\.vscode\extensions\yzane.markdown-pdf-1.4.1\node_modules\highlight.js\lib\languages\python.js
c:\Users\Fowler\AppData\Local\GitHubDesktop\app-2.2.2\resources\app\highlighter\mode\python.js
c:\Users\Fowler\AppData\Local\GitHubDesktop\app-2.2.3\resources\app\highlighter\mode\python.js
c:\Users\Fowler\AppData\Local\Google\Chrome\User Data\Default\Extensions\ngkhgikojglcgnckopipfdajaifmmnnc\4.1.34_0\python.js
c:\Users\Fowler\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu18.04onWindows_79rhkp1fndgsc\LocalState\rootfs\etc\apparmor.d\abstractions\python
c:\Users\Fowler\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu18.04onWindows_79rhkp1fndgsc\LocalState\rootfs\usr\share\bash-completion\completions\python
c:\Users\Fowler\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu18.04onWindows_79rhkp1fndgsc\LocalState\rootfs\usr\share\bash-completion\helpers\python
c:\Users\Fowler\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu18.04onWindows_79rhkp1fndgsc\LocalState\rootfs\usr\share\sosreport\sos\plugins\python.py
c:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\aniso8601\builders\python.py
c:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\python.py
c:\Users\Fowler\Documents\vscodeProjects\playarea\.venv\Scripts\python.exe
c:\Windows\Installer\$PatchCache$\Managed\8B9C64EBE8DD53846B6846E46A14F5EE\3.7.2150\python.exe
c:\Windows\Installer\$PatchCache$\Managed\9CB0624238F6F8F469EAD6566412DD7F\3.7.2150\python.exe

PS C:\Users\Fowler> where.exe /r c:\ pytest
c:\Users\Fowler\AppData\Roaming\Python\Python38\Scripts\pytest.exe
c:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\pytest.py

And finally! Whew, in case this sheds any light, here is a picture showing the python process having fun eating up my CPU during pytest.py...

I would be oh, so grateful for any assistance or thoughts!

Maybe you have some kind of conflict or strange interaction between different packages in your global Python, which is why your stuff is working fine in a venv. Maybe try progressively installing more of your global packages in a venv and testing until you figure out the packages which cause the hang? — nneonneo, Nov 26 '19 at 03:38
Good and obvious idea, except... I just did a pip list to create a requirements.txt file. I then created a venv. I activated it. I installed the full list from requirements.txt (with versions for the packages) and it STILL works in the venv??? It still fails outside the virtual env. — Paul Fowler, Nov 26 '19 at 05:09
Hmm, wacky. This appears to possibly be a Windows-specific issue since `atomicwrites` is only used on Windows now, and I lack a Windows machine to test anything. Based on the traceback it looks like `pytest` might be trying to write a lot of files. You could try for example editing `"C:\Users\Fowler\AppData\Roaming\Python\Python38\site-packages\_pytest\assertion\rewrite.py", line 274` to add a print statement to see what it's trying to write. (You could also use something like procmon to watch Python and see what it's doing?). — nneonneo, Nov 26 '19 at 05:12
The problem is in `tempfile.py`. I case of an access violation (like temp. file is named like a temp directory) it repeats endless. First issue is, why has someone set the repeat count to 2**32 ... When setting the loop to 20 tries, it still fails. At second, tempfile tries to create the file in EVERY `__pycache__` directory of every installed 3rd party package, like `colorama`, `toml`, ... FTW!?! At third I disabled the retries in case of access violation, then pytest fails in 5 out of 10 runs ... The question is now how to fix the Python core and with whom to discuss this problem? — Paebbels, Dec 21 '21 at 22:24

nevervh · Answer 1 · 2022-07-06T08:55:19.607

2

You can just run pytest from priviledged (admin) cmd for the first time. For me the issue was fixed and pytest runs from ordinary cmd now without hanging.

edited Jul 06 '22 at 08:55

answered Jul 06 '22 at 08:04

nevervh

21
3

score 1 · Answer 2 · answered Nov 26 '19 at 05:47

Fixed.

The answer appears to be

Uninstall python via the windows apps and features
Remove the c:\program files\python38 directory
Remove the ..\AppData\Roaming\Python directory
Reinstall

Not sure what the "root" problem was, but a total wipe of python fixed it. Note that the python windows installer does not remove enough python stuff.

Thank you @nneonneo for getting me thinking in the right direction.

Kevin Ding · Answer 3 · 2020-12-24T01:55:27.403

0

I have the similar problem

after i install pytest-cov or allure-pytest, pytest will hang

if i uninstall them , everything is back to normal

I have tried to install different pytest version, It didn't help

but, I tried the method you provide(uninstall python, remove the diretories), it works~, great！

but, when I install allure-pytest again, it fails again.

finally, I found that, I can't install package in privilege cmd window

If I install them in privilege cmd windows, it will make pytest hang!

so, if anyone encounter the problem I have, you can try my way.

edited Dec 24 '20 at 01:55

answered Dec 23 '20 at 14:19

Kevin Ding

568
5
9

This is not an answer - it may be more appropriate for a comment. – tilde Dec 23 '20 at 14:45
2

removing pytest-cov fixed it also for me. – Max Schmitt May 22 '21 at 09:34

score 0 · Answer 4 · edited Nov 02 '21 at 21:07

0

In my case, the package anyio, a dependency of JupyterLab, was causing this problem. It was installed at the administrator level. I uninstalled it and reinstalled at the user level and pytest started working again.

edited Nov 02 '21 at 21:07

Asclepius

57,944
17
167
143

answered Aug 15 '21 at 23:21

user1462309

480
2
10

I've just run into the same issue. Incredible, I would have never tracked this to some random unrelated package. – Wenzel Jakob Dec 12 '21 at 22:40

score 0 · Answer 5 · answered May 13 '22 at 11:35

0

Using Ubuntu 18.04/20.04 + Python 3.8.2

My tests were hanging in a similar matter after showing results.

Removing pytest-cov didn't help.

What helped was moving from Python 3.8.2 to a newer version (like 3.8.12)

answered May 13 '22 at 11:35

d.lime

393
4
15

score 0 · Answer 6 · answered Nov 30 '22 at 13:34

0

I ran into the same issue. In my case my IDE didn't load the correct conda environment. So make sure you're in the right environment for your application and use conda activate myEnv to change it if necessary.

answered Nov 30 '22 at 13:34

fehrlich

2,497
2
26
44

Python pytest hangs. For instance, "pytest --version" simply hangs

6 Answers6

Linked