It is my understanding that if PYTHONUNBUFFERED is set to a non-empty string, no buffering of stdout is supposed to happen. I use PyCharm, and by default this value is set to 1, and I have not changed it. There are no quotes around it, but in context with the other environmental settings, the 1 at least seems to be a string. I created a script in a repo / folder intended for exploration. Once I got it working well enough, I copied it over to the folder where I intended to use it.
This script has a lot of logging.info() commands to help me monitor progress. However, when I run the script in the intended repo / folder, stdout is now empty. When the script has run through all my data, the variables of the last item only show up in PyCharm's variables window, but stdout is still empty. stderr still works as expected. I have tried adding sys.stdout.flush() as described here, but that has made no difference. Aside from this issue, the script performs as expected in both places and thus is not reproduced here.
The git repo of the intended folder is 3.8 while the exploratory one is 3.9. I looked at the 3.9 changelog, and the only reference to PYTHONUNBUFFERED I see there is
bpo-13601: By default, sys.stderr is line-buffered now, even if stderr is redirected to a file. You can still make sys.stderr unbuffered by passing the -u command-line option or setting the PYTHONUNBUFFERED environment variable.
I looked at Python is not working in unbuffered mode but
- I am on Python 3.8, Ubuntu 20.04 and PyCharm 2020.2.5,
- I'd prefer an actual solution, not a hack of the builtin print()
- Even if said hack worked, how would I apply it to logging.info() instead of print()?
I found this article and here is the result:
import sys
print(sys.stdout.buffer)
<_io.FileIO name='<stdout>' mode='wb' closefd=False>
sys.__stdout__
Out[13]: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
sys.__stdout__.buffer
Out[14]: <_io.FileIO name='<stdout>' mode='wb' closefd=False>
import io
isinstance(sys.__stdout__.buffer, io.BufferedWriter)
Out[16]: False
Since the author references pytest as capturing output, I went here...
https://docs.pytest.org/en/7.1.x/how-to/capture-stdout-stderr.html#captures
where the very first line is "During test execution any output sent to stdout and stderr is captured". I have not been executing any tests. Furthermore, neither of the two repos mentioned have pytest in them, but there is a site-wide pytest. The pytest docs page also says:
You can influence output capturing mechanisms from the command line:
pytest -s # disable all capturing
But this means actually running pytest, albeit with capturing disabled. I'm not trying to run any tests here.
Are there other Python packages known to capture / buffer stdout?
How do I get my stdout back? Thanks.
EDIT / UPDATE: logging.basicConfig(level=logging.DEBUG)
