1

I want to use windows powershell pipe to process binary data sequentially, like

PS {PATH}> python p1.py | python p2.py

but it failed in my test.

p1.py

import os
import sys 
from struct import pack

p = pack('f', 3.14) 
# I want to write 3.14 as a float (4 bytes) to stdout

if sys.platform == 'win32':
    import msvcrt
    msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
    os.write(sys.stdout.fileno(), p)
else: 
    # used in linux subsystem
    outf = os.fdopen(sys.stdout.fileno(), 'wb')
    outf.write(p)
    outf.close()

p2.py

import os
import sys 
from struct import pack

# read 4 bytes from stdin and translate it to a float number

stdinfile = os.fdopen(sys.stdin.fileno(), 'rb')
rd = stdinfile.read(4)
v = unpack('<f', rd)[0]
print(v) 

If I run python p1.py | python p2.py in powershell, I will get 1.4979228, not a value closed to 3.14. But it works fine in linux bash if I run same command in linux subsystem.

Moreover, I try to abandon pipe in powershell and use a real file to store bytes. Surprisingly, it is also OK.

Furthermore, I try to re-run pipe version program in CMD. It is also OK!!!

So why I can not use binary stdout in powershell pipe? This is confusing.

Chaserwind
  • 11
  • 1
  • My python in Windows is 3.7.8 win32 version and python in linux subsystem is 3.9.5 version – Chaserwind Dec 23 '21 at 04:50
  • You didn't use `msvcrt.setmode` in your `p2.py`? Why? – Tim Roberts Dec 23 '21 at 04:50
  • I just tried it. It does not fix this problem. I still get 1.4979228. :( – Chaserwind Dec 23 '21 at 04:55
  • In short: As of PowerShell 7.2, output from external programs is invariably decoded _as text_ before further processing, which means that _raw byte output_ can neither be passed on via `|` nor captured with `>`. See the [linked duplicate](https://stackoverflow.com/q/59110563/45375) for details. The workaround is to call your external program via `cmd /c` (Windows) / `sh -c` (Unix-like platforms) and use _their_ `|` and `>` operators. – mklement0 Dec 23 '21 at 05:15
  • I got it! Thank you! – Chaserwind Dec 23 '21 at 05:26

0 Answers0