3

A while ago I switched from Enthought's old EPD to their newer Canopy system. For the most part it's nice, but one aspect has been particularly vexing.

Whenever I run a python script, either from within the Canopy iPython environment or from the command line, none of my print statements actually get printed right away when that part of the script is hit. Instead, multiple prints seem to get executed all at once at a later time.

As an example...

import numpy as np

print "About to start long computation..."
a = np.random.randn(1e8)
print "Computation finished."

doesn't print the first statement until after a is finished being generated, when both statements are printed simultaneously. (You can tell when the calculation is occurring by watching the CPU monitor.)

Does anyone know what's going on here? If relevant, I'm running Canopy 1.0.0.1160, with Python 2.7.3 64bit on a Windows 7 machine.

jfs
  • 399,953
  • 195
  • 994
  • 1,670
Blas
  • 77
  • 2
  • 6
  • In case it wasn't clear, I never had this problem with the old EPD. – Blas Sep 10 '13 at 02:03
  • 3
    Just to narrow down where the problem is: if you do `import sys` and then call `sys.stdout.flush()` after the print statement, does it appear immediately? – Danica Sep 10 '13 at 02:09
  • It does indeed. This must be the problem somehow, but is there a way to force a flush after every `print`, and why doesn't that happen automatically with Canopy like it does with every other python distribution ever? – Blas Sep 10 '13 at 02:12
  • Yeah, they must have set up default output buffering or something. There are a variety of options to force flushing available at http://stackoverflow.com/questions/107705/python-output-buffering. – Danica Sep 10 '13 at 02:29
  • 1
    Most Pythons I've used buffer writes to `stdout`. – kindall Sep 10 '13 at 02:30
  • @kindall It's typically set to line-buffering for files in text mode [as `stdout` is by default], so that a plain `print` statement would flush the buffer. – Danica Sep 10 '13 at 02:55
  • @Dougal This doesn't flush the buffer unless you run python with -u. – Hyperboreus Sep 10 '13 at 03:36

2 Answers2

3

This looks like buffered output. Try running your script as:

python -u yourscript

The -u flag turns buffering off.

(Replace python by your OS's python executable's name.)

Hyperboreus
  • 31,997
  • 9
  • 47
  • 87
2

No, this is not a change between EPD and Canopy. While I suppose there might be some python distributions which default to buffering off, EPD was not one of them -- the performance hit could have been too severe (as kindall's comment mentions.) Better to let the programmer decide when it's important for the user to see console output immediately (typically for status updates).

BTW, IPython in the Canopy GUI is simply IPython QtConsole. If you are depending on console I/O, you may also need to be aware of this longstanding issue with QtConsole:

I don't think there's a reasonable workaround for Canopy IPython, other than do it "properly", i.e. with flush.

https://support.enthought.com/entries/22157050-Canopy-Python-prompt-QtConsole-Can-t-run-interactive-OS-shell-commands

Jonathan March
  • 5,800
  • 2
  • 14
  • 16
  • Thanks Jonathan. For any who come later, I ended up just aliasing `python` to `python -u` for scripts run from the command line; haven't yet decided whether it's worth it to force Canopy to do the same. I could have sworn this was never a problem with EPD, though... – Blas Sep 10 '13 at 18:34