I am receiving an error from psycopg2 library ImportError: dlopen(/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-darwin.so, 2): Symbol not found: _PQencryptPasswordConn when I browse to an endpoint that uses SQLAlchemy to retrieve list of users from PostgreSQL database via flask API endpoint.
I am using pip-tools to manage dependencies in a flask app, including psycopg2 and psycopg2-binary on Mac OS Catalina environment.
dev.in
-r requirements.txt
black
flake8
isort
pytest
requirements.in
Flask
Flask-SQLAlchemy-Session
flasgger
marshmallow-jsonapi
marshmallow-sqlalchemy
psycopg2
psycopg2-binary
uwsgi
Generate dependencies
$ pip-compile --generate-hashes requirements.in
$ pip-compile --generate-hashes dev.in
$ pip-sync dev.txt
Run the flask app
$ pyenv activate flaskapp
$ FLASK_APP=app:create_app FLASK_ENV=development flask run
* Serving Flask app "app:create_app" (lazy loading)
* Environment: development
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 115-237-904
When curling to an endpoint to list users this error is raised in console:
127.0.0.1 - - [14/Apr/2021 16:15:29] "GET /users HTTP/1.1" 500 -
Traceback (most recent call last):
File "/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/flask/cli.py", line 184, in find_app_by_string
app = call_factory(script_info, attr, args)
File "/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/flask/cli.py", line 119, in call_factory
return app_factory()
File "/Users/simon/Development/Python/projects/flaskapp/app/__init__.py", line 33, in create_app
engine = get_engine(settings)
File "/Users/simon/Development/Python/projects/flaskapp/app/database.py", line 7, in get_engine
return engine_from_config(settings, prefix)
File "/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 733, in engine_from_config
return create_engine(url, **options)
File "<string>", line 2, in create_engine
File "/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 298, in warned
return fn(*args, **kwargs)
File "/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 548, in create_engine
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 753, in dbapi
import psycopg2
File "/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/psycopg2/__init__.py", line 51, in <module>
from psycopg2._psycopg import ( # noqa
ImportError: dlopen(/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-darwin.so, 2): Symbol not found: _PQencryptPasswordConn
Referenced from: /Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-darwin.so
Expected in: /usr/lib/libpq.5.dylib
in /Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-darwin.so
The python factory method module is:
import os
import flaskapp.config
from flasgger import Swagger
from flask import Flask
from flask_sqlalchemy_session import flask_scoped_session
from flaskapp.app.database import get_engine, get_session_factory
def _getConfig():
"""
Get Config object from FLASK_ENV
If FLASK_ENV is not set defaults to 'default'
This is used to retrieve default config object
in config.Configs.
"""
flaskEnv = os.environ.get("FLASK_ENV")
return flaskapp.config.Configs.get(flaskEnv, flaskapp.config.Configs["default"])
def create_app():
"""
Factory method to create the flask app
"""
app = Flask(__name__)
app.config.from_object(_getConfig())
settings = {"sqlalchemy.url": app.config["SQLALCHEMY_DATABASE_URI"]}
engine = get_engine(settings)
session = get_session_factory(engine)
dbSession = flask_scoped_session(session)
dbSession.init_app(app)
Swagger(app)
with app.app_context():
from flaskapp.app.views import user
app.register_blueprint(user.users_bp)
return app
The view for users is:
from flask import (
Blueprint,
jsonify
)
from flask_sqlalchemy_session import current_session
from flaskapp.app.models import (
user
)
from flaskapp.app.schemas.user import (
UserSchema
)
users_bp = Blueprint('users_bp', __name__)
def JsonAPI(*args, **kwargs):
"""Wrapper around jsonify that sets the Content-Type of the response to
application/vnd.api+json.
"""
response = jsonify(*args, **kwargs)
response.mimetype = "application/vnd.api+json"
return response
@users_bp.route('/users')
def list():
usr = current_session.query(user.UserModel).all()
data = UserSchema(many=True).dump(usr)
return JsonAPI(data)
The pip dependencies list in the virtual environment is:
pip list
Package Version
------------------------ --------
appdirs 1.4.4
attrs 20.3.0
black 20.8b1
click 7.1.2
flake8 3.9.0
flasgger 0.9.5
Flask 1.1.2
Flask-SQLAlchemy-Session 1.1
greenlet 1.0.0
iniconfig 1.1.1
isort 5.8.0
itsdangerous 1.1.0
Jinja2 2.11.3
jsonschema 3.2.0
MarkupSafe 1.1.1
marshmallow 3.11.1
marshmallow-jsonapi 0.24.0
marshmallow-sqlalchemy 0.24.2
mccabe 0.6.1
mistune 0.8.4
mypy-extensions 0.4.3
packaging 20.9
pathspec 0.8.1
pep517 0.10.0
pip 21.0.1
pip-tools 6.1.0
pluggy 0.13.1
psycopg2 2.8.6
psycopg2-binary 2.8.6
py 1.10.0
pycodestyle 2.7.0
pyflakes 2.3.1
pyparsing 2.4.7
pyrsistent 0.17.3
pytest 6.2.3
PyYAML 5.4.1
regex 2021.4.4
setuptools 49.2.0
six 1.15.0
SQLAlchemy 1.4.7
toml 0.10.2
typed-ast 1.4.3
typing-extensions 3.7.4.3
uWSGI 2.0.19.1
Werkzeug 1.0.1
wheel 0.34.2
I have seen a similar post here. Tried removing virtual environment and reinstalling. The same error occurs. Tried installing earlier version 2.7.7 but it fails to install.
Is there a stable alternative to psycopg2 ono macOS environments?
Is pyscopg2 compatible and stable with macOS?
$ pip install psycopg2==2.7.7
ERROR: Command errored out with exit status 1:
command: /Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/h5/k56b8xzj3cbbx4b3_rnmq2_00000gn/T/pip-install-orr7u5g0/psycopg2_b4923bc7e9d74723a4697a504917d633/setup.py'"'"'; __file__='"'"'/private/var/folders/h5/k56b8xzj3cbbx4b3_rnmq2_00000gn/T/pip-install-orr7u5g0/psycopg2_b4923bc7e9d74723a4697a504917d633/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/h5/k56b8xzj3cbbx4b3_rnmq2_00000gn/T/pip-record-nrhk5c5o/install-record.txt --single-version-externally-managed --compile --install-headers /Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/include/site/python3.8/psycopg2
cwd: /private/var/folders/h5/k56b8xzj3cbbx4b3_rnmq2_00000gn/T/pip-install-orr7u5g0/psycopg2_b4923bc7e9d74723a4697a504917d633/
Complete output (81 lines):
running install
running build
running build_py
creating build
creating build/lib.macosx-10.15-x86_64-3.8
creating build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/_json.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/extras.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/errorcodes.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/tz.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/_range.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/_ipaddress.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/__init__.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/psycopg1.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/extensions.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/sql.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
copying lib/pool.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2
creating build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_transaction.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/dbapi20.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_extras_dictcursor.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_with.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_types_basic.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_bug_gc.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_module.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_psycopg2_dbapi20.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_async.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_dates.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_async_keyword.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/testutils.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_connection.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_copy.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_bugX000.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/__init__.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_cursor.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_types_extras.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_sql.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_fast_executemany.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_green.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_ipaddress.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_cancel.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_quote.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/testconfig.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_errcodes.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_replication.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_lobject.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/test_notify.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
copying tests/dbapi20_tpc.py -> build/lib.macosx-10.15-x86_64-3.8/psycopg2/tests
Skipping optional fixer: buffer
Skipping optional fixer: idioms
Skipping optional fixer: set_literal
Skipping optional fixer: ws_comma
running build_ext
building 'psycopg2._psycopg' extension
creating build/temp.macosx-10.15-x86_64-3.8
creating build/temp.macosx-10.15-x86_64-3.8/psycopg
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION=2.7.7 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=100001 -DHAVE_LO64=1 -I/Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/include -I/Users/simon/.pyenv/versions/3.8.2/include/python3.8 -I. -I/Library/PostgreSQL/10/include -I/Library/PostgreSQL/10/include/postgresql/server -c psycopg/psycopgmodule.c -o build/temp.macosx-10.15-x86_64-3.8/psycopg/psycopgmodule.o
psycopg/psycopgmodule.c:689:18: error: incomplete definition of type 'struct _is'
while (interp->next)
~~~~~~^
/Users/simon/.pyenv/versions/3.8.2/include/python3.8/pystate.h:20:8: note: forward declaration of 'struct _is'
struct _is;
^
psycopg/psycopgmodule.c:690:24: error: incomplete definition of type 'struct _is'
interp = interp->next;
~~~~~~^
/Users/simon/.pyenv/versions/3.8.2/include/python3.8/pystate.h:20:8: note: forward declaration of 'struct _is'
struct _is;
^
2 errors generated.
It appears you are missing some prerequisite to build the package from source.
You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.
For further information please check the 'doc/src/install.rst' file (also at
<http://initd.org/psycopg/docs/install.html>).
error: command 'clang' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/h5/k56b8xzj3cbbx4b3_rnmq2_00000gn/T/pip-install-orr7u5g0/psycopg2_b4923bc7e9d74723a4697a504917d633/setup.py'"'"'; __file__='"'"'/private/var/folders/h5/k56b8xzj3cbbx4b3_rnmq2_00000gn/T/pip-install-orr7u5g0/psycopg2_b4923bc7e9d74723a4697a504917d633/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/h5/k56b8xzj3cbbx4b3_rnmq2_00000gn/T/pip-record-nrhk5c5o/install-record.txt --single-version-externally-managed --compile --install-headers /Users/simon/.pyenv/versions/3.8.2/envs/flaskapp/include/site/python3.8/psycopg2 Check the logs for full command output.
