1

Before anyone tells me, I completely realize what I'm trying to do is absurd but I have my reasons.

I have a server on which I can write a script (bash or python). The script needs to accept binary data from STDIN and save it to a file. To be precise, the binary data is a WAV file.

What I am doing is using an iOS app called Workflow. The app will let me record an audio file and then send it as STDIN input to a script that it will run over SSH. So my script needs to take that STDIN input and save it to a file.

Any ideas/thoughts? I'm completely lost on where to start.

IMTheNachoMan
  • 5,343
  • 5
  • 40
  • 89
  • 2
    the tool is called [`cat`](http://linux.die.net/man/1/cat) – Jakuje Dec 28 '15 at 20:27
  • To write binary from stdin with bash : http://unix.stackexchange.com/questions/19043/how-can-i-handle-raw-binary-data-in-a-bash-pipe – Quentin Dec 28 '15 at 20:28
  • related: [Reproduce the unix cat command in python](http://stackoverflow.com/q/11532980/4279). Here's [`cat.py` in Python 3](https://gist.github.com/zed/cda879d141081e5764bd) – jfs Dec 28 '15 at 21:36
  • `cat - > $file` to be exact, @Jakuje This will copy stdin to a file, even if it's binary. – doekman Dec 26 '17 at 18:42

2 Answers2

5

To save stdin to a file on a POSIXy system:

  1. make sure stdin is opened in the binary mode
  2. check whether fadvise improves the time performance in your case e.g., see copyfileobj() in contrib
  3. read chunks from stdin and write them to the file

In general, you could also try os.sendfile() if stdin might be redirected from a file (mmap-able file descriptor) (it doesn't apply in your case).

#!/usr/bin/env python
import os
import shutil
import sys

with os.fdopen(sys.stdin.fileno(), 'rb') as input_file,\
     open(sys.argv[1], 'wb') as output_file:
    shutil.copyfileobj(input_file, output_file)

Example:

$ python -msave_stdin output_file

that should be equivalent to:

$ cat >| output_file
Community
  • 1
  • 1
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • Note that this closes the file descriptor of STDIN which could lead to issues. To avoid it, don't open "stdin" using the "with" context but do: `input_file = os.fdopen(sys.stdin.fileno(), 'rb')` – famzah Jul 30 '20 at 21:47
  • right, the copy is done successfully but STDIN (file descriptor 0) is closed. Execute `os.system(f'ls -la /proc/{os.getpid()}/fd')` before and after the block, and you will see my point. A (long running) program with closed FD=0 could experience strange things. One example is that if you open a file afterwards, it could get FD=0, because it's free. And if we're not careful and assume that FD=0 is always "stdin", we may get confused. – famzah Jul 31 '20 at 20:06
3

To read from stdin in python : 

import sys
input = sys.stdin.read()

To write binary to a file:

with open('name_of_file_here','wb') as output:
   output.write(input)

Where 'wb' sends for write, in binary mode.

Sidervs
  • 61
  • 4
  • I should have mentioned, the read() function will read everything coming from stdin, until it encounters EOT, which is \x04 (in hex) or Ctrl+D. To stay safe you can check how many bytes you want to pipe into stdin, and then use sys.stdin.read(number_of_bytes) – Sidervs Dec 28 '15 at 20:58
  • are you talking about Windows? Piping binary data on Windows is a non-trivial task e.g., see [what happens in PowerShell](http://stackoverflow.com/a/33959798/4279). Piping binary data is trivial on POSIX (it doesn't stop on `\x04` or anything else -- don't confuse what you type in terminal (that might be affected by the terminal settings) and the data that is piped via application's stdin). – jfs Dec 28 '15 at 21:33
  • I was talking about POSIX. Just tested it, and you are absolutely right. Not sure why I thought that... Thanks! – Sidervs Dec 28 '15 at 21:46
  • to avoid corrupting newlines (if `os.linesep != b'\n'`) or for Python 3 compatibility, open stdin in binary mode. To support large files, read stdin in chunks. – jfs Dec 28 '15 at 23:54