5

INTRODUCTION:

It is a web application environment using Python 3.2, NGINX and uWSGI.

Python 3.2, NGINX and uWSGI components was installed via build and installation ("./configure", "make" and "make install") and they use OpenSSL 1.0.X also installed via build and installation.

The application is running on Ubuntu 20.04 LTS.

PROBLEM:

The application cannot start and the uWSGI log shows the information below...

Thu Jan 21 21:44:37 2021 - Respawned uWSGI worker 4 (new pid: 99572)
Thu Jan 21 21:44:37 2021 - worker 3 buried after 0 seconds
Thu Jan 21 21:44:37 2021 - worker 7 buried after 0 seconds
Thu Jan 21 21:44:37 2021 - worker 1 buried after 0 seconds
Thu Jan 21 21:44:37 2021 - worker 8 buried after 0 seconds
Thu Jan 21 21:44:37 2021 - worker 5 buried after 0 seconds
Thu Jan 21 21:44:37 2021 - received message 0 from emperor
Thu Jan 21 21:44:37 2021 - SIGINT/SIGQUIT received...killing workers...
Thu Jan 21 21:44:37 2021 - mem-collector thread started for worker 4
Thu Jan 21 21:44:37 2021 - !!! uWSGI process 99543 got Segmentation Fault !!!
Thu Jan 21 21:44:37 2021 - *** backtrace of 99543 ***
/usr/local/lb/uwsgi_ve32/bin/uwsgi(uwsgi_backtrace+0x2e) [0x556d323f077e]
/usr/local/lb/uwsgi_ve32/bin/uwsgi(uwsgi_segfault+0x27) [0x556d323f0b67]
/lib/x86_64-linux-gnu/libc.so.6(+0x46210) [0x7f60a0f33210]
/lib/x86_64-linux-gnu/libc.so.6(+0x186b7e) [0x7f60a1073b7e]
/usr/local/ssl/lib/libcrypto.so.1.0.0(+0x12254c) [0x7f60a147054c]
/usr/local/ssl/lib/libcrypto.so.1.0.0(lh_insert+0x56) [0x7f60a14707e6]
/usr/local/ssl/lib/libcrypto.so.1.0.0(OBJ_NAME_add+0x74) [0x7f60a13bbc84]
/lib/x86_64-linux-gnu/libssl.so.1.1(+0x36392) [0x7f60a0067392]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x1247f) [0x7f60a181547f]
/lib/x86_64-linux-gnu/libcrypto.so.1.1(CRYPTO_THREAD_run_once+0xd) [0x7f609ff3d78d]
/lib/x86_64-linux-gnu/libssl.so.1.1(OPENSSL_init_ssl+0x5b) [0x7f60a006757b]
/lib/x86_64-linux-gnu/libpq.so.5(+0x26304) [0x7f60a00ea304]
/lib/x86_64-linux-gnu/libpq.so.5(PQconnectPoll+0xe80) [0x7f60a00d4ec0]
/lib/x86_64-linux-gnu/libpq.so.5(+0x11fd7) [0x7f60a00d5fd7]
/lib/x86_64-linux-gnu/libpq.so.5(PQconnectdb+0x38) [0x7f60a00d9048]
/usr/local/lb/lbg_ve32/lib/python3.2/site-packages/psycopg2/_psycopg.cpython-32m.so(+0x10939) [0x7f60a012e939]
/usr/local/lb/lbg_ve32/lib/python3.2/site-packages/psycopg2/_psycopg.cpython-32m.so(+0x117a2) [0x7f60a012f7a2]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0xab58c) [0x7f60a11c558c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyObject_Call+0x5b) [0x7f60a1174dcb]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(_PyObject_CallFunction_SizeT+0xd2) [0x7f60a1175192]
/usr/local/lb/lbg_ve32/lib/python3.2/site-packages/psycopg2/_psycopg.cpython-32m.so(+0xbace) [0x7f60a0129ace]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x76e3) [0x7f60a1210b53]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0x81f92) [0x7f60a119bf92]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyObject_Call+0x5b) [0x7f60a1174dcb]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0xfb9) [0x7f60a120a429]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0x81f92) [0x7f60a119bf92]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyObject_Call+0x5b) [0x7f60a1174dcb]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0xfb9) [0x7f60a120a429]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x65f1) [0x7f60a120fa61]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x670c) [0x7f60a120fb7c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0x81eaf) [0x7f60a119beaf]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyObject_Call+0x5b) [0x7f60a1174dcb]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0x7033c) [0x7f60a118a33c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyObject_Call+0x5b) [0x7f60a1174dcb]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0xacf47) [0x7f60a11c6f47]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0xab58c) [0x7f60a11c558c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyObject_Call+0x5b) [0x7f60a1174dcb]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x4106) [0x7f60a120d576]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x670c) [0x7f60a120fb7c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x670c) [0x7f60a120fb7c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x65f1) [0x7f60a120fa61]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x65f1) [0x7f60a120fa61]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x670c) [0x7f60a120fb7c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x65f1) [0x7f60a120fa61]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x670c) [0x7f60a120fb7c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x65f1) [0x7f60a120fa61]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0x81f92) [0x7f60a119bf92]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyObject_Call+0x5b) [0x7f60a1174dcb]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0xfb9) [0x7f60a120a429]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x65f1) [0x7f60a120fa61]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalFrameEx+0x670c) [0x7f60a120fb7c]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyEval_EvalCodeEx+0x733) [0x7f60a12115e3]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(+0x81eaf) [0x7f60a119beaf]
/usr/local/lb/py32/lib/libpython3.2m.so.1.0(PyObject_Call+0x5b) [0x7f60a1174dcb]
*** end of backtrace ***

IMPORTANT:

Appears that the application is "mixing" OpenSSL 1.0.X components with OpenSSL 1.1.X components...

/usr/local/ssl/lib/libcrypto.so.1.0.0(+0x12254c) [0x7f60a147054c]
/usr/local/ssl/lib/libcrypto.so.1.0.0(lh_insert+0x56) [0x7f60a14707e6]
/usr/local/ssl/lib/libcrypto.so.1.0.0(OBJ_NAME_add+0x74) [0x7f60a13bbc84]
/lib/x86_64-linux-gnu/libssl.so.1.1(+0x36392) [0x7f60a0067392]
/lib/x86_64-linux-gnu/libcrypto.so.1.1(CRYPTO_THREAD_run_once+0xd) [0x7f609ff3d78d]
/lib/x86_64-linux-gnu/libssl.so.1.1(OPENSSL_init_ssl+0x5b) [0x7f60a006757b]

I really need help with this question... I have tried everything!

Thanks! =D

Eduardo Lucio
  • 1,771
  • 2
  • 25
  • 43
  • 1
    Is there a reason you must use OpenSSL 1.0? If not you can install uWSGI using `sudo -H pip3 install uwsgi` or `sudo apt install uwsgi`. – Selcuk Jan 22 '21 at 01:17
  • 1
    It is a legacy application and all of its components use OpenSSL 1.0.X. It seems to me that some configuration is missing during the compilation process so that uWSGI consumes "/usr/local/ssl/lib/libssl.so.1.0.0" (OpenSSL 1.0.X) and not "/usr/lib/x86_64-linux-gnu/libssl.so.1.1 "(OpenSSL 1.1.X). Thanks! =D – Eduardo Lucio Jan 22 '21 at 01:24
  • @Selcuk See this thread that addresses a very similar problem: https://stackoverflow.com/a/54253374/3223785 . =D – Eduardo Lucio Jan 22 '21 at 01:43
  • 1
    That makes sense. This might be a weird suggestion but is there any chance that you can run the system on a distro with OpenSSL 1.0.x (such as Ubuntu 16.04) instead? – Selcuk Jan 22 '21 at 01:46
  • @Selcuk Unfortunately not, as the client asked to update to the latest version of Ubuntu (20.04 LTS). Thanks! =D – Eduardo Lucio Jan 22 '21 at 01:49
  • 1
    Well, you may try running a prebuilt uWSGI then. Possibly one that was compiled on an Ubuntu 16 machine. Sorry, segfaults are hard to debug. – Selcuk Jan 22 '21 at 01:50
  • @Selcuk Yes! That is precisely why I am so "lost"... =| – Eduardo Lucio Jan 22 '21 at 01:51

1 Answers1

5

CAUSE: Most modern versions of Ubuntu Server have PostgreSQL 9.4 configured to use ssl - tls actually - in a version above version 1.0 which is incompatible with Python 3.2 that uses OpenSSL to have the ssl resource. When Python 3.2 tries to connect to PostgreSQL 9.4 using ssl, some library in use by the first (psycopg2?) tries to load a more modern version of OpenSSL compatible with the client, then the application starts to operate with two different versions of OpenSSL - one of which is incompatible with Python 3.2 - which causes a segmentation fault error in the application load.

SOLUTION: We can use PostgreSQL 9.4 with a 1.0 version of ssl or disable this feature. We chose disable "ssl". The procedure below configures PostgreSQL 9.4 to do not use ssl to establish connections as was done in the older versions of distros that use PostgreSQL 9.4.

PROCEDURE:

Open the file /var/lib/pgsql/9.4/data/postgresql.conf - the path may vary - and comment out these lines:

ssl = on # (change requires restart)

... , ...

ssl_cert_file = '/etc/ssl/certs/ssl-cert-snakeoil.pem' # (change requires restart)

... and...

ssl_key_file = '/etc/ssl/private/ssl-cert-snakeoil.key' # (change requires restart)

Restart the Python 3.2 application and PostgreSQL 9.4.

For PostgreSQL 9.4 the following commands can be used to restart it...

systemctl stop postgresql
sleep 2
systemctl start postgresql

Done! Thanks! =D

Eduardo Lucio
  • 1,771
  • 2
  • 25
  • 43