extra line while printing process output

Question

I have the following python3 code to print difference of two files/directories:

def out_diff(arg1, arg2):
    out = subprocess.Popen(['diff', '-r', arg1, arg2], 
               stdout=subprocess.PIPE, 
               stderr=subprocess.STDOUT)
    stdout, stderr = out.communicate()
    print(type(stdout))
    print(type(stderr))
    if len(stdout) != 0:
        print('stdout: ', arg1, arg2, stdout.decode()),  # error line
    if stderr is not None:
        print('stderr: ', arg1, arg2, stderr)
    print('end of function')

out_diff('output1', 'output2')

And its output is the following:

stdout:  output1 output2 diff: output1: No such file or directory
diff: output2: No such file or directory

end of function

My question is, whether or not I put a trailing comma on the stdout printing line (marked by "#"), an empty line is always printed. I don't understand why that is happening. Something should have changed based on the presence of the trailing comma.

My expectation was that since a '\n' was in the stdout string, then if I did NOT put a trailing comma then 2 empty lines would be printed.

score 1 · Answer 1 · answered Jan 07 '19 at 04:57

1

In Python 3, print is a function that always returns None. By adding a comma after the call, you create and immediately discard the tuple (None,) instead of just None. This makes no difference in your code whatsoever, besides slowing it down imperceptibly.

To suppress the newline that print normally appends to its output, you will need to pass in the keyword-only argument end. end defaults to \n. Set it to '' instead:

print('stdout: ', arg1, arg2, stdout.decode(), end='')

As you pointed out correctly, stdout.decode() will most likely already contain a trailing newline.

answered Jan 07 '19 at 04:57

Mad Physicist

107,652
25
181
264

Thanks for your answer. But I accepted the other one because it was more detailed, especially it pointed out my mistake of adding comma at the end which was a python2 feature, and advised to use run instead. – R71 Jan 07 '19 at 06:05
@R71. No worries. I appreciate the feedback. – Mad Physicist Jan 07 '19 at 06:10

tripleee · Accepted Answer · 2019-01-07T05:57:26.787

1

It's not clear what you are trying to accomplish here or what you are actually asking.

In Python 3 a trailing comma after print() doesn't do anything. You are probably confusing this with the behavior of print (without parentheses) in Python 2, where a trailing comma said to omit the final newline.

In any event, redirecting standard error to standard output and then examining both standard output and standard error seems like a very confused thing to do.

Moreover, you really should not use Popen when run will do.

def out_diff(arg1, arg2):
    out = subprocess.run(['diff', '-r', arg1, arg2], 
               stdout=subprocess.PIPE, 
               stderr=subprocess.PIPE,   # Notice how this is distinct
               check=False,              # Be explicit that we tolerate errors from diff
               universal_newlines=True)  # Decode text
    print(type(out.stdout))
    print(type(out.stderr))
    if len(out.stdout) != 0:
        print('stdout: {0} {1} {2}'.format(arg1, arg2, out.stdout), end='')
    if len(out.stderr) != 0:
        print('stderr: {0} {1} {2}'.format(arg1, arg2, stderr), end='')
    print('end of function')

Notice how print(string, end='') will omit the final newline in Python 3.

Anyway, the proper way to interact programmatically with diff is to examine its exit status (out.returncode) instead of its output.

edited Jan 07 '19 at 05:57

answered Jan 07 '19 at 05:01

tripleee

175,061
34
275
318

For (much) more on how to use `subprocess` properly, see https://stackoverflow.com/questions/4256107/running-bash-commands-in-python/51950538#51950538 – tripleee Jan 07 '19 at 05:14
Thanks very much. You caught several errors in my code at one shot. Can you add a brief explanation on the check=False, I did not quite understand that. – R71 Jan 07 '19 at 06:09
`check=True` says to fail and raise an exception if the subprocess returns a non-zero exit code. I added `check=False` just to make it explicit that we really don't want that (`diff` will return "failure" if there are differences). The linked answer explains this in a lot more detail. – tripleee Jan 07 '19 at 06:10
BTW, out.returncode != 0 only if stderr, eg when a file is missing. If both files exist and there is a diff, then outcode is still 0, so we also have to check for len(out.stdout)!=0. – R71 Jan 07 '19 at 06:14
That's not my experience. `diff` returns `out.returncode = 1` if both files exist and there is a difference. Are you on some wacky platform like Windows? – tripleee Jan 07 '19 at 06:18

extra line while printing process output

2 Answers2