3

As the jenkins user, my sqlite_version is 3.7.17

bash-4.2$ python3.8
Python 3.8.2 (default, May  8 2020, 12:44:28) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3 
>>> sqlite3.sqlite_version
'3.7.17'

as my root user the sqlite_version is 3.31.1

[root@jenkins ~]# python3.8
Python 3.8.2 (default, May  8 2020, 12:44:28) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.31.1'

They are both using the same python binary

[root@jenkins ~]# which python3.8
/usr/local/bin/python3.8

bash-4.2$ which python3.8
/usr/local/bin/python3.8

Steps I took after installing new sqlite3 from source:

  • Deleted the old version of sqlite3 in /bin
  • I added /usr/local/bin to the jenkins path in /var/lib/jenkins/.bashrc

Do you have any idea how to make the jenkins python use the new sqlite3 version?

Update:

Running sqlite3.__file__:

Jenkins

>>> sqlite3.__file__
'/usr/local/lib/python3.8/sqlite3/__init__.py'

Root

>>> sqlite3.__file__
'/usr/local/lib/python3.8/sqlite3/__init__.py'
tread
  • 10,133
  • 17
  • 95
  • 170

2 Answers2

2

Following works for me on CentOS 7.7:


Install Python Dependencies:

sudo yum -y groupinstall 'Development Tools'
sudo yum -y install openssl-devel bzip2-devel libffi-devel

SQLite source install:

wget https://sqlite.org/2020/sqlite-autoconf-3310100.tar.gz
tar -xf sqlite-autoconf-3310100.tar.gz
cd sqlite-autoconf-3310100/
./configure
make
sudo make install

Python source install:

wget https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tar.xz
tar -xf Python-3.8.2.tar.xz
cd Python-3.8.2/
sudo LD_RUN_PATH=/usr/local/lib ./configure --enable-optimizations
sudo LD_RUN_PATH=/usr/local/lib make altinstall

Test:

[centos@jenkins ~]$ sudo su
[root@jenkins centos]# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[root@jenkins centos]# grep PATH /root/.bashrc
export PATH=$PATH:/usr/local/bin
[root@jenkins centos]# python3.8
Python 3.8.2 (default, May 15 2020, 07:26:39)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.31.1'
>>> sqlite3.__file__
'/usr/local/lib/python3.8/sqlite3/__init__.py'
>>>

[root@jenkins centos]# exit
[centos@jenkins ~]$
[centos@jenkins ~]$ sudo su - jenkins
Last login: Fri May 15 07:44:53 UTC 2020 on pts/0
-bash-4.2$ id
uid=996(jenkins) gid=993(jenkins) groups=993(jenkins) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
-bash-4.2$ pwd
/var/lib/jenkins
-bash-4.2$ grep PATH .bashrc
grep: .bashrc: No such file or directory
-bash-4.2$ python3.8
Python 3.8.2 (default, May 15 2020, 07:26:39)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.31.1'
>>> sqlite3.__file__
'/usr/local/lib/python3.8/sqlite3/__init__.py'
>>>
Technext
  • 7,887
  • 9
  • 48
  • 76
  • 1
    Aha, so `LD_RUN_PATH` is used for link time resolution. Where I was using `LD_LIBRARY_PATH` which specifies runtime resolution of linked libraries. – tread May 15 '20 at 08:20
1

Did you check the PYTHONPATH and/or LD_LIBRARY_PATH for both root and jenkins ?

About Python path (PYTHONPATH)

You can use sys.path to print the effective PYTHONPATH for each environment. If you see a difference, you are probably using 2 different versions of sqlite3 library.

If it's indeed the issue, you can try to change the PYTHONPATH in your jenkins recipe.

About shared library path (LD_LIBRARY_PATH//etc/ld.so.conf)

It could also be a wrong PATH for the dynamic loader. Indeed .so files could be loaded from different path. To change/see the shared library path, you can refer to the following answer: https://unix.stackexchange.com/a/22999

LD_LIBRARY_PATH is one way to configure shared library path but you could also use /etc/ld.so.conf. There is an answer about how to set the library path for Jenkins here

Raphael Medaer
  • 2,528
  • 12
  • 18
  • @surfer190 do you have feedback about my answer ? – Raphael Medaer May 14 '20 at 19:22
  • The `sys.path` are the same in both cases. It seems the issue is to do with `ld` the GNU linker. If this is set `LD_LIBRARY_PATH=/usr/local/lib`, the correct sqlite3 version is used. – tread May 15 '20 at 08:14
  • Hi surfer190. I edited my answer to add few references about shared library path and Jenkins. Hope it helps. Indeed there was not only the _python path_ but also the _shared library path_! In some cases some users might have also the issue of _python path_ so I let this option in the answer! – Raphael Medaer May 15 '20 at 08:29