Getting a "(core dumped) python -m pytest" error running pytest in Jenkins when same test runs and passes on the command line

Question

I'm running some tests using on Python 3 code in Jenkins and am getting a (core dumped) python -m pytest error but when I run the same test on my command line the test passes.

Here is my setup.

• Python: 3.6
• Pytest 5.3.4
• OS on Jenkins server: Ubuntu 18.04

The contents of my test script are as follows:

#!/bin/bash
. venv/bin/activate
whoami
python -m pytest

When run the test script from the command line I get the following:

(venv) [Wed Mar 04 15:48:20] bill@billc-dev:~/PycharmProjects/SLAM-ER_STE$ ./regtest.sh
bill
======================================================================================================================= test session starts ========================================================================================================================
platform linux -- Python 3.6.9, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
PyQt5 5.13.2 -- Qt runtime 5.13.2 -- Qt compiled 5.13.2
rootdir: /home/bill/PycharmProjects/SLAM-ER_STE
plugins: shutil-1.7.0, qt-3.3.0
collected 6 items                                                                                                                                                                                                                                                  

Scripting/steps2py_test.py ...                                                                                                                                                                                                                               [ 50%]
UserInterface/login_test.py ...                                                                                                                                                                                                                              [100%]

======================================================================================================================== 6 passed in 0.74s =========================================================================================================================
(venv) [Wed Mar 04 15:48:39] bill@billc-dev:~/PycharmProjects/SLAM-ER_STE$ 

But when I run the same test script in Jenkins I get the following:

15:13:20 [test1] $ /bin/sh -xe /tmp/jenkins7917398407283219031.sh
15:13:20 + ./regtest.sh
15:13:20 jenkins
15:13:20 ============================= test session starts ==============================
15:13:20 platform linux -- Python 3.6.9, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
15:13:20 rootdir: /var/lib/jenkins/workspace/test1
15:13:21 ./regtest.sh: line 4: 19362 Aborted                 (core dumped) python -m pytest
15:13:21 Build step 'Execute shell' marked build as failure
15:13:21 Finished: FAILURE

I'm sure that this is some sort of config issue on the Jenkins server but I'm at a loss to know what it could be. Any help would be greatly appreciated.

Answer 1

QUICK: To solve the Ubuntu 20.04 specific issue, it seems to involve a missing xinerama library, which can be fixed with:

sudo apt-get install libxcb-xinerama0

BACKGROUND: For what it's worth, I discovered the above by putting 'strace' in front of the python call. Huge amount of output, the tail end of which is an annoying one-char-per-log-line log of the final exception report. But if you scroll up to above that and look around, you can see the system trying very very hard to locate the xcb-xinerama library. A Google search later and I had the detail I needed. PyQt5's pip install info apparently doesn't list the xcb-xinerama requirement because it's "always been there", but newer Ubuntu maybe has removed it.

ADDITIONAL: I just discovered the Xvfb plugin for Jenkins which can help with this. It removes (or can remove) the need to export DISPLAY and sort out your xhost issues, and it probably works better if you have multiple projects, multiple executors, etc.

1. Add the Xvfb plugin
2. Install Xvfb ('sudo apt-get install xfvb' for Debian/Ubuntu/Mint)
3. Add a tool for the plugin that specifies the installation directory (any name you want, "default" is apparently common). It was installed in /usr/bin for me.
4. In the configuration for your project, in the "Build Environment" you can turn on Xvfb for the build, and the Advanced button gives you other options. You can add the "-ac" additional option to Xvfb to disable access control, which may ease initial setup.

The above has allowed me to run other simple X-based apps inside the build, but (on Ubuntu 20.04 * Python3.8.5 * pytest-6.1.2) I'm still getting core dumps trying to run the tests that require Qt to work. Have tried QT_DEBUG_PLUGINS=1 which did not provide any additional info. So, ultimately, I have the same problem. This was working on Ubuntu 18.04, but suddenly stopped working after upgrade to Ubuntu 20.04 (I'm not saying it's an Ubuntu issue, just that that was the trigger point in time...)

Answer 2

Actually we figured out what was happening. The software being tested uses PyQt5 and as a result is trying to draw GUI images but Jenkins did not know where to draw them, leading to the core dump issue.

What fixed this was doing the following two things:

1. Giving the jenkins user permission to access the x11 server by running $>sudo xhost +si:localuser:jenkins
2. In the bash script add export DISPLAY=:0.0 to tell jenkins where to look for the Xserver. Bash script now looks like this:

#!/bin/bash
. venv/bin/activate
whoami
export DISPLAY=:0.0
python -m pytest