Python write to Linux device using subprocess gives ValueError : embedded null byte

Question

On a Debian Linux system I have a Python script that prepares a string to be written out to a USB CDC device at /dev/ttyACM0. This write occurs at the end of the script as follows...

USBpipe = open("/dev/ttyACM0", 'w')
shellCmd = subprocess.Popen(["echo", USBpacket], stdout = USBpipe)
USBpipe.close()

...where USBpacket is a string. If I make this a pure string such as using USBpacket = "test" the code executes correctly and I've verified the data appears on the USB device. However, during normal execution USBpacket gets bytes appended to it via the chr() function, and some of these may be zero. When this happens I get this error running the script:

Traceback (most recent call last):
  File "/root/gpstoPIC.py", line 247, in <module>
    shellCmd = subprocess.Popen(["echo", USBpacket], stdout = USBpipe)
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1756, in _execute_child
    self.pid = _posixsubprocess.fork_exec(
ValueError: embedded null byte

I've tried a couple of solutions that haven't worked, such as doing wb instead of w on the open of /dev/ttyACM0, and using bytes() to convert from string to binary. What is the correct way to write out this data?

Why are you using `echo` instead of `USBpipe.write(USBpacket)`? Wouldn't that be simpler? — wjandrea, Mar 30 '22 at 21:44
It's a holdover from my research on looking up "how to send data to USB device Linux command line", `echo` was the only thing that I got to work, so when I was ready to write a script and not just test direct from the shell that's what I used. However, after reading your comment I ditched using `subprocess` and used `write()` instead and it was just fine. Since this was stated in a comment and not an official answer I picked the other post here as an answer even though it repeats what you said (along with some extraneous information). — Christopher Theriault, Apr 04 '22 at 21:38

score 1 · Accepted Answer · edited Apr 04 '22 at 22:12

The shell in general, and echo in particular, can't cope with arbitrary binary input. Lucky thing then that Python can, and so you don't need a subprocess here at all.

with open("/dev/ttyACM0", "wb") as usb_pipe:
    usb_pipe.write(USBpacket)

If you really wanted to use a subprocess, try cat (but then that's really a useless use of cat). Notice also how the open uses "wb" to open the filehandle in binary mode (plain "w" applies an encoding which again is not tolerant of or compatible with arbitrary binary data). And yes, if the data you want to send is really binary, USBpacket should indeed be a bytes object.

Finally, perhaps notice how I preferred snake_case over CamelCase for local scalar variable names; perhaps USBpacket should be renamed to adhere to PEP-8 recommendations, too.

Perhaps see also http://nedbatchelder.com/text/unipain.html for an exposition of the differences between strings and `bytes`. It covers Python 2, too, which is increasingly irrelevant, but apparently close to your mental model. — tripleee, Apr 02 '22 at 18:59

Python write to Linux device using subprocess gives ValueError : embedded null byte

1 Answers1