1

I would like to run the following commands directly in Python:

dumpcap -q -f http -i eth0 -w - | tshark -l -n -T json -r - | my_app.py

I would like to run it by using subprocess and asyncio for having it run in async.

So at first I would like to run:

dumpcap -q -f http -i eth0 -w -

The output from this should be piped into the next command, which should/could run unsynchronized:

tshark -l -n -T json -r -

The output from this should be piped into a stream which I can work with.

Is there a straightforward solution for this?

Alfred Balle
  • 1,135
  • 4
  • 16
  • 32

1 Answers1

-1

In addition to @user4815162342's answer, note that you can simply pass the full shell command to create_subprocess_shell and use pipes to communicate with both ends of the subprocess:

Example:

proc = await asyncio.create_subprocess_shell(
    "tr a-z A-Z | head -c -2 | tail -c +3",
    stdin=asyncio.subprocess.PIPE,
    stdout=asyncio.subprocess.PIPE,
)
stdout, _ = await proc.communicate(b"**hello**")
assert stdout == b"HELLO"
Vincent
  • 12,919
  • 1
  • 42
  • 64
  • `proc.communicate()` will attempt to read all of the data instead of streaming line-by-line. this is bad for memory use (imagine if you expect the child stdout to be larger than main RAM) and bad for performance (processing the first line won't happen until the child has finished) – inetknght Jul 01 '20 at 05:35
  • 1
    @inetknght Sure, it really depends on the command being run. For instance, `communicate()` is fine if the command already includes some aggregation and produces a small output (e.g `tail -n 1`, `wc`, etc.). – Vincent Jul 01 '20 at 07:15