0

We need to use some of the newest sqlite feature like "UPDATE FROM", which is available from sqlite 3.33.0 (https://www.sqlite.org/changes.html).

In current debian docker image we use stable (buster) distribution, which has older version of sqlite lib (the name is libsqlite3-dev).

I compiled sqlite3 locally from sources in .sh script, but I'm not able to configure pythons sqlite3 module to use the just compiled version of sqlite.

This is how the code looks like:

# Download newer libsqlite3-dev with "UPDATE FROM" clause support
cd data_processing
wget "https://www.sqlite.org/2020/sqlite-amalgamation-3340000.zip" -O sqlite.zip
unzip sqlite.zip
rm sqlite.zip
mv sqlite-amalgamation-3340000 sqlite-3.34
cd sqlite-3.34
gcc shell.c sqlite3.c -lpthread -ldl -o sqlite3
# remove no longer needed file (everything else than just created sqlite3)
find . ! -name 'sqlite3' -type f -exec rm -f {} +
cd -
export PATH="$(pwd)/my_app/sqlite-3.34:$PATH"

Here we can see, that adding the binary to the path is not enough. I also tried adding:

sys.path.insert(
    0,
    '/home/user/project/libs/my_app/sqlite-3.34'
)

into python code, no success. Is there any way to configure sqlite3 python module to use a different sqlite lib installed on the system?

wbrycki
  • 121
  • 1
  • 8
  • Look at this question, and see if that might help you: https://stackoverflow.com/questions/14541869/how-to-upgrade-sqlite3-in-python-2-7-3-inside-a-virtualenv – JohanL Dec 04 '20 at 19:10

2 Answers2

1

Use binary package of pysqlite3: https://github.com/coleifer/pysqlite3 in your requirements, add

pip install pysqlite3-binary

in python code, use pysqlite3:

import pysqlite3
    (...) conn = pysqlite3.connect(r"filename") (...)

instead of sqlite3.

Alternative, as Anthony suggested: swapping the system binary You can swap the binary of the system:

import sys
sys.modules['sqlite3'] = __import__('pysqlite3')

And the you don't have to use pysqlite3 in your code.

wbrycki
  • 121
  • 1
  • 8
1

to add to answer above (not enough rep)

import sys
sys.modules['sqlite3'] = __import__('pysqlite3')

to swap in the binary in-place of system sqlite3

credit: https://romandc.com/zappa-django-guide/walk_database/

Anthony
  • 71
  • 1
  • 2