Django/Sqlite3 Module Load Path in Python Virtual Environment

Question

I've developed a website in Django (3.0.8) using the latest Python version (3.8.3). I'm working on a Unix server with Python 3.6.9 installed, which includes Sqlite version 3.7.17. Django apparently requires Sqlite 3.8 or higher.

So, I compiled a local copy of the latest Python following the guide here: https://www.a2hosting.com/kb/developer-corner/python/using-a-newer-version-of-python

I set up a virtual environment as described above and everything is working smoothly, except that Python is still using the old Sqlite version. After hours of work trying to figure all this out I'm stumped.

I can access Python and Sqlite3 via command line in standard environment:

-bash-4.2$ python --version
Python 3.6.9

-bash-4.2$ python
>>> import sqlite3, inspect
>>> sqlite3.sqlite_version
'3.7.17'

>>> inspect.getfile(sqlite3)
'opt/rh/rh-python36/root/usr/lib64/python3.6/sqlite3/__init__.py'

and in virtual environment:

-bash-4.2$ source bin/venv/bin/activate
(venv) -bash-4.2$ python --version
Python 3.8.3

-bash-4.2$ python
>>> import sqlite3, inspect
>>> sqlite3.sqlite_version
'3.7.17'

>>> inspect.getfile(sqlite3)
'users/.../lib/python3.8/sqlite3/__init__.py

So, despite running Python 3.8.3 and pointing to the correct library (as far as I can tell) in the virtual environment, the sqlite version is still the same as the standard environment. Any suggestions would be greatly appreciated!

Answer 1

SQlite3 is part of a python installation and is not an external library, i.e. you cannot upgrade using a package manager like pip. With each python version you will have a different Sqltie version. However, this is not true if you have replaced the dll files. I have seen some users do this on windows and to me this is rather a risky approach.

If you have replaced the dlls then try reverting that and the output of this should be distinct from python 3.6.9

-bash-4.2$ python
>>> import sqlite3, inspect
>>> sqlite3.sqlite_version
'3.7.17'  # would be different version here

You can also (if you want), download the sqlite3.dll (pre-compiled binary) file from here of a version >=Sqlite 3.8. Then copy over this file in the DLLs folder in the python installation directory. You may want to make a backup of your old sqlite3.dll file just in case you may decide to revert.

Answer 2

Okay, the answer provided by Laenka-Oss solved my problem with minor adjustments: django can't find new sqlite version? (SQLite 3.8.3 or later is required (found 3.7.17))

Install Sqlite from source:

cd ~
wget https://www.sqlite.org/2020/sqlite-autoconf-3320300.tar.gz
tar zxvf sqlite-autoconf-3290000.tar.gz
cd sqlite-autoconf-3290000

./configure --prefix=$HOME/opt/sqlite --disable-dynamic-extensions --enable-static --disable-shared
make && make install

Update library paths by adding these lines to your .bash_profile:

export PATH=$HOME/opt/sqlite/bin:$PATH
export LD_LIBRARY_PATH=$HOME/opt/sqlite/lib
export LD_RUN_PATH=$HOME/opt/sqlite/lib

Install Python from source:

cd ~
wget https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tar.xz
tar xJf Python-3.8.3.tar.xz
cd Python-3.8.3
./configure --prefix=$HOME/opt
make && make install

Now update Python path by adding this to the end of your .bash_profile:

export PATH=$HOME/opt/Python-3.8.3/bin:$PATH

Check that everything worked:

source .bash_profile
python3 --version
Python 3.8.3

python3
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.32.3'

If you encounter a "SqLite header and version mismatch: ..." error, make sure you run source .bash_profile or restart your connection. If that doesn't work, then double check the sqlite install used the commands shown above.

Your .bash_profile should look like:

export PATH=$HOME/opt/sqlite/bin:$PATH
export LD_LIBRARY_PATH=$HOME/opt/sqlite/lib
export LD_RUN_PATH=$HOME/opt/sqlite/lib
export PATH=$HOME/opt/Python-3.8.3/bin:$PATH