4

OS/Software installed:

root@TACIT admin]# cat /etc/*release*
CentOS Linux release 7.6.1810 (Core) 

root@TACIT admin]# python3.7 --version
Python 3.7.4

[root@TACIT admin]# sqlite3 --version
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa6

(T3PSA) [root@TACIT src]# django-admin --version
2.2

Software locations:

[root@TACIT admin]# which python3.7
/usr/local/bin/python3.7

[root@TACIT admin]# which sqlite3
/usr/bin/sqlite3

(T3PSA) [root@TACIT src]# which django-admin
/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/bin/django-admin

I compiled Python 3.7.4 from source and installed following these instructions without any problems:

https://tecadmin.net/install-python-3-7-on-centos/

I upgraded from Sqlite 3.7.17 to Sqlite 3.29.0 following these instructions without any problems (other than I had to install some additional ".so" libraries):

https://linuxhint.com/upgrade-to-latest-sqlite3-on-centos7/

Unfortunately Python 3.7.4 is still using the old version of Sqlite3 (3.7.17):

[root@TACIT admin]# python3.7
Python 3.7.4 (default, Aug 16 2019, 16:34:12) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version        
'3.7.17'

I'm using Django and need it to run at least v3.8.3 (see the very end of the following output):

(T3PSA) [root@TACIT src]# python manage.py makemigrations
Traceback (most recent call last):
  File "manage.py", line 21, in <module>
main()
  File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/core/management/__init__.py", line 357, in execute
django.setup()
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/apps/registry.py", line 114, in populate
app_config.import_models()
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/apps/config.py", line 211, in import_models
self.models_module = import_module(models_module_name)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/contrib/auth/models.py", line 2, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/contrib/auth/base_user.py", line 47, in <module>
class AbstractBaseUser(models.Model):
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/base.py", line 117, in __new__
new_class.add_to_class('_meta', Options(meta, app_label))
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/base.py", line 321, in add_to_class
value.contribute_to_class(cls, name)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/options.py", line 204, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/__init__.py", line 28, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/utils.py", line 201, in __getitem__
backend = load_backend(db['ENGINE'])
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/utils.py", line 110, in load_backend
return import_module('%s.base' % backend_name)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 66, in <module>
check_sqlite_version()
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 63, in check_sqlite_version
raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)
django.core.exceptions.ImproperlyConfigured: **SQLite 3.8.3 or later is required (found 3.7.17).**

I've looked at the following posts to try to figure this out but I think I don't know enough about compiling code to figure out the proper solution. Nothing I have tried from these links works.

https://superuser.com/questions/1425212/where-does-django-look-for-sqlite-instance-sqlite-3-8-3-or-later-is-required

https://unix.stackexchange.com/questions/434100/updating-the-sqlite-version-used-by-python-3-on-centos-7

django can't find new sqlite version? (SQLite 3.8.3 or later is required (found 3.7.17))

https://superuser.com/questions/1447285/python-wont-compile-with-a-new-sqlite3-on-centos-7

It could be that the instructions I'm following don't work because my software installation locations/paths are different from the users in the above posts, I'm not sure... that's why I noted "Software locations:" above. Or maybe I need to uninstall Python 3.7 somehow rather than 'reinstall over the top' of an existing installation? I installed from source, not sure if I need to do this and if so, how to go about it properly.

Any help resolving this would be greatly appreciated. Thanks in advance!

=========================================================================

This section added after getting instructions from FlipperPA

Followed instructions exactly to "Upgrade the CentOS 7 system install of SQLite to 3.29 by compiling it from source". Below is the result.

[root@TACIT src]# sqlite3 --version
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa6

I had already deleted my original source compile directory (/usr/src/Python-3.7.4) several times yesterday as I tried all of the other articles from the original post. So starting from scratch, I did this:

cd /usr/src
tar -zxvf Python-3.7.4.tgz
cd Python-3.7.4
./configure --enable-optimizations
make altinstall
make clean
make install

[root@TACIT Python-3.7.4]# python3.7
Python 3.7.4 (default, Aug 17 2019, 08:20:39) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.7.17'  

Python is still using the wrong version of SQLite. I did the 'make altinstall' command because CentOS 7.6 comes with Python 2.7.5 by default and I read that I shouldn't mess with that. So 'make altinstall' installs Python 3.7.4 in another location so as to leave the default installation of python fully intact.

=========================================================================

Manually uninstalling Python 3.7.4 (installed from source)

So I did the following:

find / -name *3.7*

The resultant list of files is below. The first group of files/folders I left alone. The second group of files/folders I removed.

/var/lib/yum/yumdb/s/27f7f0189d0898d0e87007d97102619629a4de6d-sqlite-3.7.17-8.el7-x86_64
/var/lib/yum/yumdb/s/e942412b453fd568720258d674b18e7fcd06a20f-sqlite-devel-3.7.17-8.el7-x86_64
/usr/lib/python2.7/site-packages/backports.ssl_match_hostname-3.7.0.1.dist-info
/usr/lib/python2.7/site-packages/configparser-3.7.3.dist-info
/usr/lib/python2.7/site-packages/flake8-3.7.7.dist-info
/usr/share/doc/sqlite-3.7.17
/usr/share/doc/git-1.8.3.1/RelNotes/1.5.3.7.txt

/usr/bin/easy_install-3.7
/usr/bin/pip3.7
/usr/bin/python3.7m
/usr/bin/python3.7
/usr/bin/python3.7m-config
/usr/bin/pydoc3.7
/usr/bin/idle3.7
/usr/bin/2to3-3.7
/usr/bin/pyvenv-3.7
/usr/lib/pkgconfig/python-3.7.pc
/usr/lib/python3.7
/usr/lib/python3.7/lib2to3/Grammar3.7.4.final.0.pickle
/usr/lib/python3.7/lib2to3/PatternGrammar3.7.4.final.0.pickle
/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu
/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu/libpython3.7m.a
/usr/lib/libpython3.7m.a
/usr/share/man/man1/python3.7.1
/usr/include/python3.7m
/usr/local/bin/easy_install-3.7
/usr/src/Python-3.7.4.tgz
/usr/src/Python-3.7.4
/usr/src/Python-3.7.4/Doc/whatsnew/3.7.rst
/usr/src/Python-3.7.4/build/lib.linux-x86_64-3.7
/usr/src/Python-3.7.4/build/temp.linux-x86_64-3.7
/usr/src/Python-3.7.4/build/temp.linux-x86_64-3.7/usr/src/Python-3.7.4
/usr/src/Python-3.7.4/build/scripts-3.7
/usr/src/Python-3.7.4/build/scripts-3.7/pydoc3.7
/usr/src/Python-3.7.4/build/scripts-3.7/idle3.7
/usr/src/Python-3.7.4/build/scripts-3.7/2to3-3.7
/usr/src/Python-3.7.4/build/scripts-3.7/pyvenv-3.7
/usr/src/Python-3.7.4/libpython3.7m.a

I then did the following steps:

tar -zxvf Python-3.7.4.tgz
cd Python-3.7.4
./configure --prefix=/usr --enable-optimizations
make
make altinstall
[root@TACIT Python-3.7.4]# python3.7
Python 3.7.4 (default, Aug 17 2019, 16:34:42) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3    
>>> sqlite3.sqlite_version
'3.7.17'

Not sure what else I can do here.

Dan
  • 331
  • 6
  • 17
  • How do you install Python packages? Do you use `pip`? – John Gordon Aug 17 '19 at 00:18
  • CentOS 7.6 comes with Python 2.7.5. Using PIP, I installed “pipenv”, then installed Python 3.7.4 from source as described above. Then I did “mkdir/cd T3PFA” and ran command “pipenv —python 3.7 install django=2.2”. This created a virtual environment which I launch with “pipenv shell”. That’s when you see the “django-admin —version” output above. Then I “mkdir/cd src ” and created my django project “django-admin startproject t3psa .” followed by “python manage.py makemigrations” which resulted in my error. So I used pipenv to create my virtual environment and install django. – Dan Aug 17 '19 at 14:02

3 Answers3

2

UPDATE 2020-10-03:

I've compiled a version of _sqlite.so which you can drop at the root of your Python 3.8 project:

https://github.com/FlipperPA/django-s3-sqlite/raw/master/shared-objects/python-3-8/_sqlite3.so

If you drop this at the root of a Python or Django project, it should be picked up when Python's SQLite 3 wrapped attempts to _import sqlite, overriding this module: https://github.com/python/cpython/tree/b6f5b9dd21a37fa97af38521941025259f1556e0/Modules/_sqlite

Good luck!

ORIGINAL MESSAGE:

This is an issue with the included version of SQLite on CentOS 7 being quite old. Here's how I solved it, IIRC:

  • Upgrade the CentOS 7 system install of SQLite to 3.29 by compiling it from source. Download the source, and then be sure to ./configure --prefix=/usr to upgrade the system version. Then make and sudo make install. You must include the prefix or it'll install to /usr/local instead by default!
  • Make sure the system version installed is 3.29 with sqlite3 --version
  • Re-compile Python 3.7.x. Be sure to do a make clean before sudo make install if you've already installed it.
  • Open python3.7 and check the version. import sqlite3, followed by sqlite3.sqlite_version

That should work; the only difference is I installed from Python 3.6 from IUS Community. After upgrading the system SQLite, I did a sudo yum remove python36u then a sudo yum install python36u and the Python included SQLite had successful upgraded to 3.29. Good luck!

FlipperPA
  • 13,607
  • 4
  • 39
  • 71
  • Awesome. I will give this a shot and report back. Thank you for posting! – Dan Aug 17 '19 at 14:56
  • Added section to original post: "This section added after getting instructions from FlipperPA". SQLite3 upgrade worked. Python recompile didn't. I must have done something wrong. – Dan Aug 17 '19 at 15:41
  • Hmmm, there may be another step. Before compiling Python 3.7 again, let's try `sudo make uninstall`, and then `./configure`, `make clean`, and `sudo make install`. I'm thinking something in the compilation process is cached from the previous install. – FlipperPA Aug 17 '19 at 19:14
  • sudo make uninstall: “make: *** No rule to make target ‘uninstall’. Stop.” Also, when I get back to “sudo make install” again, I’m going to “make altinstall”, right? – Dan Aug 17 '19 at 21:43
  • Also tried “./configure”, “make clean”, “make altinstall” with same results. – Dan Aug 17 '19 at 21:50
  • Unfortunately, it looks like uninstalling will be a bit more difficult that I thought. See here: https://unix.stackexchange.com/questions/190794/uninstall-python-installed-by-compiling-source Hopefully, you can do something similar for Python 3.7 to uninstall the `altinstall` version. `sudo make altinstall` should be fine, as long as you're explicitly calling `python3.7` to run it. But I think we're going to have to figure out how to uninstall before it'll take the new version. – FlipperPA Aug 17 '19 at 22:17
  • See new section "Manually uninstalling Python 3.7.4 (installed from source)" in original post. Not sure what else I can do here. – Dan Aug 17 '19 at 23:41
  • Can you do a `find /usr -name "python3.7"`? I'm wondering if it has installed in more than one spot. I'm hoping someone who knows more about compiling Python from source than me might have some suggestions. If we don't get an answer by Monday, I'll fire up a fresh CentOS 7 install on a virtual machine and see if I can get it figured. – FlipperPA Aug 18 '19 at 00:11
  • /usr/bin/python3.7 & /usr/lib/python3.7 (first one is a file, second one is a directory, both of which I removed above before recompile). Thank you brother, I appreciate your help and responsiveness. – Dan Aug 18 '19 at 00:33
2

I think that OP must fix the path to sqlite3 while installing Python:

sudo LD_RUN_PATH=/usr/lib ./configure --enable-optimizations
sudo LD_RUN_PATH=/usr/lib make altinstall
help-ukraine-now
  • 3,850
  • 4
  • 19
  • 36
  • That did the trick. I had read about this in another post but wasn't sure if I should do it or not. One question so I understand. Why when sqlite3 is in /usr/bin do I point to /usr/lib in the commands above? – Dan Aug 21 '19 at 16:58
0

change

def check_sqlite_version():
if Database.sqlite_version_info < (3, 8, 3):
    raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)

to:

def check_sqlite_version():
if Database.sqlite_version_info < (3, 7, 17):
    raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)
Fethi Pounct
  • 1,059
  • 5
  • 6
  • File : "..../home/....../.local/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 67, in check_sqlite_version raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version) django.core.exceptions.ImproperlyConfigured: SQLite 3.8.3 or later is required (found 3.7.17). – Fethi Pounct Apr 15 '21 at 16:54
  • This one conflicts with another script and throws another error. – Sohan Arafat Jul 30 '21 at 15:04