Python subprocess using perl for formatting is giving incomplete output

Question

I'm having an issue reading output from a python subprocess command.

The bash command from whose output I want to read:

pacmd list-sink-inputs | tr '\n' '\r' | perl -pe 's/ *index: ([0-9]+).+?application\.process\.id = "([^\r]+)"\r.+?(?=index:|$)/\2:\1\r/g' | tr '\r' '\n'

When I run this via bash I get the intended output:

4 sink input(s) available.
6249:72
20341:84
20344:86
20350:87

When I try to get it's output via python's subprocess running either one :

1. subprocess.Popen(cmnd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()[0].decode('UTF-8')

2. check_output(cmnd,shell=True).decode('UTF-8')

3. subprocess.run(cmnd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE).stdout.decode('utf-8')

where cmnd = """pacmd list-sink-inputs | tr '\n' '\r' | perl -pe 's/ *index: ([0-9]+).+?application\.process\.id = "([^\r]+)"\r.+?(?=index:|$)/\2:\1\r/g' | tr '\r' '\n'"""

It gives the following output:

'4 sink input(s) available.\n\x02:\x01\n\x02:\x01\n\x02:\x01\n\x02:\x01\n'

Which is unintended as it doesn't have the 6249:72 ,etc. numbers I want. Even stderr is blank and returncode is 0 as intended.

The only workaround, I could find was to redirect the bash output to a text file and then read the text file via python which I don't want to use because that's unnecessary file IO.

I've already gone through Missing output from subprocess command, Python Subprocess Grep, Python subprocess run() is giving abnormal output [duplicate] and many others but can't wrap my head around what's going wrong.

Answer 1

You have a quoting issue. """\1""" means chr(0o1). To produce the string \1, you could use """\\1""". The other instances of \ should be \\ as well.

Since all instances of \ need to be escaped, you could also use r"""\1""".

Other issues:

• \1 and \2 outside of a regular expression is wrong anyways. You should be using $1 and $2.

• There's no use for a mutliline literal here. "..." or r"..." would suffice.

• The whole tr business can be avoided by using -0777 to cause perl to treat the entire file as one line.

This gives us:

cmnd = "pacmd list-sink-inputs | perl -0777pe's/ *index: (\\d+).+?application\\.process\\.id = "([^\\n]+)"\\n.+?(?=index:|$)/$2:$1\\n/sag'"

or

cmnd = r"pacmd list-sink-inputs | perl -0777pe's/ *index: (\d+).+?application\.process\.id = "([^\n]+)"\n.+?(?=index:|$)/$2:$1\n/sag'"

But why is Perl being used at all here? You could easily do the same thing in Python!