If I use the Python boto3 library to run a command on an AWS ECS container using execute_command
as demonstrated here, and my command is simply echo -ne '\n'
(a single, explicit newline character), when I read from the websocket stream that has my command's output, the output is b'\r\n'
.
This isn't a huge deal for commands that produce text output, and can be easily fixed, but if I run a command that produces binary output, this corrupts my data. Although this can be fixed by replacing \r\n
with \n
on the client when I read from the websocket stream, another thing that happens is that the bytes 0xfd
, 0xfe
, and 0xff
are not transferred at all. That's a lot harder to fix, since there's no way to know where those bytes were removed from.
The only workaround I have found is to run the command as bash -c "echo -en '\n' | base64 -w0"
, and decode it back to binary after receiving it. This works, but makes the output ~33% larger, which, for large binary data like what I'm dealing with, increases the download time by minutes.
What is making these changes? Is there a way to disable this? If not, is there a more efficient workaround then base64?