Jenkins not printing output of python script in console

Question

I have a python script(myscript.py) as follows:

#!/bin/python
import os
import optparse
import subprocess
import sys
sys.stdout.flush()

print("I can see this message on Jenkins console output")
cmd="sshpass -p 'xxx' ssh test@testmachine 'cmd /c cd C:\stage && test.bat'"
retval=subprocess.call(cmd,shell=True)
print retval

In jenkins, I have a job with execute shell as follows:

#!/bin/sh
./myscript.py

Problem: Jenkins console shows only "I can see this message on Jenkins console output". If there is any output from the subprocess call, it does not print it out on the console.

If I putty to Server A and run the same command (./myscript.py) on shell, I can see the output of subprocess call.

How can I print this output of subprocess call on Jenkins console?

FYI: As you can see from my command, the subprocess call is running a batch file on windows; Jenkins is running on Linux; There is ssh setup between the two machines..

Edit: My test.bat looks like this:

echo off
RMDIR /S /Q C:\Test

IF %ERRORLEVEL% NEQ 0 (
  ECHO Could not delete
  EXIT /b %ERRORLEVEL%
)

if I run this batch file locally on windows server, it returns a 1 ( because am holding a file open in Test folder )

But when the python script calls this batch file using the subprocess call, all i get is a Zero for retval.

Why is this and how to fix this? If I can capture the correct retval, I can make the Jenkins job fail.

Edit 12/12: Helllo!! Anybody! Somebody! Help!

I suspect your problem is the use of sshpass. This will set up its own tty to fool ssh into allowing an interactive session. It also sets a return code based on the success of the connection, not the result of the command run inside ssh. Can you try using ssh directly, authorizing with a public key instead? — Peter Brittain, Dec 10 '16 at 11:45
No luck! I created a public key and got rid of ssh pass.. I still get the same result... :-( — user1164061, Dec 12 '16 at 21:35
Did you also check the return code? With sshpass removed, ssh should now return the exit coce of the executed command. — Peter Brittain, Dec 13 '16 at 01:22
If that doesn't fix it, the only other explanation I can think of is that you've hit this bug in windows: http://stackoverflow.com/questions/11137702/batch-exit-code-for-rd-is-0-on-error-as-well — Peter Brittain, Dec 13 '16 at 01:36
I checked exit code of the executed command. It shows 0. Can you please add all of this as an answer to my question? I know I dont have a solution to my issue.. but at least I can grant the bonus for the person who helped the most! — user1164061, Dec 13 '16 at 19:54
@PeterBrittain: the link that you shared solved my problem. I added an || to RMDIR. Now it is returning the errorlevel correctly ! Awesome find! Thank you so much. Please add this as your answer to this question. You get the 50 bonus points!!! — user1164061, Dec 13 '16 at 21:29

score 13 · Answer 1 · answered Dec 12 '16 at 20:51

13

I wonder if it has to do anything with stdout being buffered Can you try setting PYTHONUNBUFFERED before running your command?

export PYTHONUNBUFFERED=true

answered Dec 12 '16 at 20:51

Illusionist

5,204
11
46
76

score 9 · Answer 2 · answered Feb 11 '21 at 05:21

9

In my Jenkins environment, executing python scripts with the unbuffered argument makes the output appear immediately. Like this:

python3 -u some_script.py

More information comes from the help menu (python3 --help):

-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x

answered Feb 11 '21 at 05:21

Wayne Workman

449
7
14

1

Thanks, after adding this parameter, everything works as expected. – Lancer.Yan Apr 06 '21 at 17:56

Billy · Answer 3 · 2016-12-08T17:00:38.047

3

When you execute your script in a shell, Python sets your shell's STDOUT as the subprocess's STDOUT, so everything that gets executed gets printed to your terminal. I'm not sure why, but when you're executing in Jenkins the subprocess is not inheriting the shell's STDOUT so its output is not displayed.

In all likelihood, the best way to solve your problem will be to PIPE the STDOUT (and STDERR for good measure) and print it after the process ends. Also, if you exit with the exit code of your subprocess and the exit code is not 0, it will likely terminate your Jenkins job.

p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
                     stderr=subprocess.PIPE, shell=True)
exit_code = p.wait()  # wait for it to end
print('Got the following output from the script:\n', p.stdout.read().decode())
print('Got the following errors from the script:\n', p.stderr.read().decode())
print('Script returned exit code:', exit_code)

sys.exit(exit_code)

edited Dec 08 '16 at 17:00

answered Dec 08 '16 at 16:55

Billy

5,179
2
27
53

Thanks for your help.But this is not solving the problem still. I have a text file open within C:\Test; So the return code should be 1 from my test.bat. But this is what i get : ('Got the following errors from the script:\n', u'') ('Script returned exit code:', 0) – user1164061 Dec 09 '16 at 17:56
@slav - Any suggestion based on your expertise? – user1164061 Dec 09 '16 at 18:01
" I have a text file open within C:\Test; So the return code should be 1 from my test.bat" that doesnt say much. you can open the same file multiple times on certain environments as long as it's for read or maybe even write. https://stackoverflow.com/a/22469791/260242 the code by billy seems correct for what i know. if it doesnt return 1 is because it ended without error. – Nande Mar 09 '23 at 02:15
https://stackoverflow.com/q/14510634/260242 multiple reads on python – Nande Mar 09 '23 at 02:21

score 3 · Accepted Answer · edited May 23 '17 at 11:54

TL; DR

The fix is to use some conditional execution (the || operator) on rmdir to fix the errorlevel being returned.

Investigation

This was a corker of a bug, with quite a few twists and turns! We initially suspected that the stdout chain was broken somehow, so looked into that through explicit use of pipes in Popen and then removing sshpass from your command and so using the output from ssh directly.

However, that didn't do the trick, so we moved on to looking at the return code of the command. With sshpass removed, ssh should return the result of the command that was run. However, this was always 0 for you.

At this point, I found a known bug in Windows that rmdir (which is the same as rd) doesn't always set errorlevel correctly. The fix is to use some conditional execution (the || operator) on rmdir to fix up the errorlevel.

See batch: Exit code for "rd" is 0 on error as well for full details.

For anyone else looking into the same issue: RMDIR /S /Q C:\Test && echo success || echo failure works i.e it returns an errorlevel but RMDIR /S /Q C:\Test &&echo success || echo %ERRORLEVEL% does not echo the errorlevel.. it has to be in the next line of code — user1164061, Dec 14 '16 at 17:28

Jenkins not printing output of python script in console

4 Answers4

TL; DR

Investigation