8

I'm trying to build Python 3.9.1 with SSL support on CentOS 7.

[myuser@server Python-3.9.1]$ which openssl
/usr/local/bin/openssl
[myuser@server Python-3.9.1]$ openssl version
OpenSSL 1.1.1g  21 Apr 2020

Running this command

sudo ./configure CPPFLAGS="-I/usr/local/openssl/include" LDFLAGS="-L/usr/local/openssl/lib" --with-ssl

followed by "make" worked for Python 3.7, but when i run the above on 3.9 and then run make I get this output

...
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_lzma                 _tkinter              _uuid              
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd                
time                                                           


Following modules built successfully but were removed because they could not be imported:
_hashlib              _ssl                                     


Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

running build_scripts
creating build/scripts-3.9
copying and adjusting /usr/src/Python-3.9.1/Tools/scripts/pydoc3 -> build/scripts-3.9
copying and adjusting /usr/src/Python-3.9.1/Tools/scripts/idle3 -> build/scripts-3.9
copying and adjusting /usr/src/Python-3.9.1/Tools/scripts/2to3 -> build/scripts-3.9
changing mode of build/scripts-3.9/pydoc3 from 644 to 755
changing mode of build/scripts-3.9/idle3 from 644 to 755
changing mode of build/scripts-3.9/2to3 from 644 to 755
renaming build/scripts-3.9/pydoc3 to build/scripts-3.9/pydoc3.9
renaming build/scripts-3.9/idle3 to build/scripts-3.9/idle3.9
renaming build/scripts-3.9/2to3 to build/scripts-3.9/2to3-3.9
/bin/install -c -m 644 ./Tools/gdb/libpython.py python-gdb.py
gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall    -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include -I/usr/local/openssl/include   -DPy_BUILD_CORE -o Programs/_testembed.o ./Programs/_testembed.c
gcc -pthread -L/usr/local/openssl/lib    -Xlinker -export-dynamic -o Programs/_testembed Programs/_testembed.o libpython3.9.a -lcrypt -lpthread -ldl  -lutil -lm   -lm 
sed -e "s,@EXENAME@,/usr/local/bin/python3.9," < ./Misc/python-config.in >python-config.py
LC_ALL=C sed -e 's,\$(\([A-Za-z0-9_]*\)),\$\{\1\},g' < Misc/python-config.sh >python-config

So evidently Python 3.9 has NOT been built with SSL. What else do I need to do to build Python 3.9 with the openssl module I have installed?

Dave
  • 15,639
  • 133
  • 442
  • 830
  • This might be because it might not be able to find your openssl libraries. What did your `./configure` say about your OpenSSL? More specifically, we are looking at line `checking whether compiling and linking against OpenSSL works...`. – Amit Singh Jan 03 '21 at 18:34
  • This is usually an issue with your `./configure` not being able to find your OpenSSL libraries and link it. – Amit Singh Jan 03 '21 at 18:37
  • 1
    Similar issue has been raised in the [Python bug issue 34028](https://bugs.python.org/issue34028) – Amit Singh Jan 03 '21 at 18:40
  • 1
    @AmitSingh, The line "checking whether compiling and linking against OpenSSL works... yes" appears. There is also a warning, "configure: WARNING: unrecognized options: --with-ssl" – Dave Jan 03 '21 at 19:00
  • 1
    The correct argument is `--with-openssl` probably. Can you try with that? – Amit Singh Jan 03 '21 at 19:01
  • Okay, sorry I missed this part. You have to specify the openssl directory for that to work. Be sure to specify it as the directory that contains the `lib` folder – Amit Singh Jan 03 '21 at 19:38
  • Gotcha. So I was able to run configure with "--with-openssl=/usr/local/openssl" but when I ran "make" after I still got the same error as reported in the question. – Dave Jan 03 '21 at 19:39
  • 1
    Can you try doing this with `yum install zlib-devel bzip2-devel tk-devel uuid-devel`? This is probably a case of missing libraries then. – Amit Singh Jan 03 '21 at 21:09
  • I ran the command you have above, then ran the configure (with openssl) command and make but still get the same "Could not build the ssl module!" message. – Dave Jan 03 '21 at 21:42
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/226790/discussion-between-amit-singh-and-dave). – Amit Singh Jan 03 '21 at 21:43
  • 1
    What if **before** *configure* (and *make*) you `export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/openssl/bin` (I assume that's your path to *libcrypto* and *libssl*)? – CristiFati Jan 03 '21 at 22:06
  • 1
    Just a note on your approach: `sudo configure` is usually wrong. You configure, build and test as regular user, only if you install system-wide you use `sudo make install` or something similar. – Ulrich Eckhardt Jan 03 '21 at 22:20
  • @ChristiFati, I have files "/usr/local/openssl/lib/libcrypto.so" and "/usr/local/openssl/lib/libcrypto.a" so should my export command be "export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/openssl/lib"? – Dave Jan 04 '21 at 00:55
  • @Dave, yes, add the path to your custom built *libcrypto.so*, and see if it works. – CristiFati Jan 04 '21 at 14:58
  • @Dave you can try this [post](https://stackoverflow.com/a/5939170/14475852) – Chandan Jan 04 '21 at 18:44

5 Answers5

6

Python3.10 source build on CentOS 7.6

The above posts were very helpful to guide me to my solution. I want to share in case someone else could use this:

First the development libs

  1. sudo yum groupinstall "Development Tools" -y
  2. sudo yum install openssl11-devel libffi-devel bzip2-devel xz-devel -y
    • for static linking use openssl11-static

Then configure

So for some reason CentOS ships openssl1.1.1 but it is installed in a way that doesn't jive with Python's configure script.

# note that we #include <openssl/foo.h>, so the OpenSSL headers have to be in
    # an 'openssl' subdirectory

    if ! $found; then
        OPENSSL_INCLUDES=
        for ssldir in $ssldirs; do
            { $as_echo "$as_me:${as_lineno-$LINENO}: checking for openssl/ssl.h in $ssldir" >&5
$as_echo_n "checking for openssl/ssl.h in $ssldir... " >&6; }
            if test -f "$ssldir/include/openssl/ssl.h"; then
                OPENSSL_INCLUDES="-I$ssldir/include"
                OPENSSL_LDFLAGS="-L$ssldir/lib"
                OPENSSL_LIBS="-lssl -lcrypto"

Since Python is including headers a subfolder is desired: $ssldir/include/openssl/ssl.h

I just created a new folder with the correct layout:

/opt/openssl111/
├── include
│   └── openssl
│       └── *.h
└── lib
    ├── libcrypto.so
    └── libssl.so

Once in place, the trifecta:

  1. ./configure --enable-optimizations --with-lto --with-openssl=/opt/openssl111/
    • lto is link time optimization
  2. make -j8
  3. make altinstall (installs in /usr/local/ not to overwrite system python)
squjd
  • 61
  • 2
  • 4
5

The issue is because of missing development libraries that should be installed. To install them, as per Python Developer installation docs for CentOS, run

sudo yum install yum-utils
sudo yum-builddep python3

Run the following to build Python

./configure
make
make test
sudo make install

In case you want to build it with your custom SSL library, pass the argument --with-openssl along with your library path. The path should contain a lib folder containing the required files.

./configure --with-openssl=<your-openssl-library-path>
Amit Singh
  • 2,875
  • 14
  • 30
  • I tried your version with the "./configure --with-openssl" command and after running "/usr/local/bin/python3 -m pip install --upgrade pip" I get a warning, "WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available" so I don't think the steps worked. – Dave Jan 04 '21 at 00:51
  • Why were you running `/usr/local/bin/python3 -m pip install --upgrade pip`? Did it get installed but is now giving you this warning? – Amit Singh Jan 04 '21 at 04:51
  • Your `make` is probably still giving errors. Can you check the logs? – Amit Singh Jan 04 '21 at 05:01
2

When you compile Python, you need to tell it which OpenSSL headers and libraries to use. Try this before running "./configure" when building Python

export CFLAGS="$CFLAGS $(pkg-config --cflags openssl11)"
export LDFLAGS="$LDFLAGS $(pkg-config --libs openssl11)"

This is the most reliable way to get the correct flags passed to the compiler and linker. It's also worth checking that there aren't any conflicting values being passed into the CFLAGS / LDFLAGS environment variables, which might stymie your efforts.

This assumes that you have the openssl-devel package installed, obviously. :-)

gjvc
  • 81
  • 2
  • 3
0

I believe this is the key:

Following modules built successfully but were removed because they could not be imported: _hashlib _ssl

What could cause this to happen is that the libraries installed in a non-standard location can't be loaded. So, locate the module (it is built, after all) and run ldd on it. See it's manpage for further detail and keep in mind that Python modules are shared objects (or DLLs on MS Windows). It will probably show a missing dependency on libssl.so. In order to resolve that, configure Linux' dynamic linker to pick up the file. I'd like to refer you to man ld.so for instructions, no use duplicating that info here.

Ulrich Eckhardt
  • 16,572
  • 3
  • 28
  • 55
-1
# try the following:
export CC="gcc"
export CXX="g++"
export CPP="cpp"
export F90="gfortran"
export F77="gfortran"
export FC="gfortran"

dnf install gcc bzip2-devel tk-devel openssl-devel uuid-devel sqlite-devel gdbm-devel readline-devel xz-lzma-compat xz-devel libffi-devel libnsl2-devel
./configure --enable-shared --enable-optimizations
make -j
make install
Oren
  • 1
  • 3
  • 2
    Thank you for answering the question. Although the code above might be a valid answer for the question, adding comments/explanations would help those looking at your answer. – ahajib Sep 08 '21 at 15:08