airflow initdb: undefined symbol: Py_GetArgcArgv

Question

I followed the documentation to install Apache-airflow. https://airflow.apache.org/docs/stable/start.html

When I execute airflow initdb, an error occurs every time.

x-MacBook-Pro:~ x$ airflow initdb
 ......
import airflow.utils.dag_processing
  File "/Library/Python/3.7/site-packages/airflow/utils/dag_processing.py", line 40, in <module>
    from setproctitle import setproctitle
ImportError: dlopen(/Library/Python/3.7/site-packages/setproctitle.cpython-37m-darwin.so, 2): Symbol not found: _Py_GetArgcArgv
  Referenced from: /Library/Python/3.7/site-packages/setproctitle.cpython-37m-darwin.so
  Expected in: flat namespace
 in /Library/Python/3.7/site-packages/setproctitle.cpython-37m-darwin.so

One answer suggested that this is a problem with the binary package. But I still don't know how to solve that. This is the link https://github.com/psycopg/psycopg2/issues/807.

MacOSX 10.15.3

pip 20.0.2

Python 3.7.3

by any chance did you resolve this? – dv akhil Mar 20 '20 at 07:39 — dv akhil, Mar 20 '20 at 07:39
@dvakhil no, it's still there. – changcui Apr 02 '20 at 14:25 — changcui, Apr 02 '20 at 14:25

score 3 · Answer 1 · answered Oct 29 '20 at 14:15

3

Of course I had the same issue. macOS 10.15.2.

I had python 3.8 globally installed. I did some research and I found out that the reason was the way that c library m-darwin.so was compiled.

This steps help me fixed the issue:

Installed pyenv

brew install pyenv

Downgraded to python 3.7.0

pyenv install 3.7.0

Set python 3.7.0 globally

pyenv global 3.7.0

Running python3 -V should give you:

Python 3.7.0

Recreated my project environment using virtualenv:

python3 -m virtualenv --python=python3.7 my awesome_env

Activated my env and installed all dependencies and it woorked.

source awesome_env/bin/activate
pip install -r requirements.txt

I think this whole process trigger the recompilation of the libraries.

answered Oct 29 '20 at 14:15

Robert Gabriel

1,151
12
24

I think the problem you had was because your `Python3.8.x` version was `x<5`. – Jorrick Sleijster Oct 06 '21 at 07:53
I don't really understand what you doing here but this worked for me and solved my issue. – Akhil Apr 21 '22 at 19:07
@Akhil, I'm not an expert on Python or C libs but I can speculate that the C library was compiled for a different python version. – Robert Gabriel Apr 25 '22 at 11:36

score 2 · Answer 2 · answered Aug 31 '20 at 19:19

2

I had the same problem, when I was using the system (OS) python3 interpreter, i.e. /usr/bin/python3

you can simply install python 3.7 or 3.8 (it works for both) with homebrew:

brew install python@3.8

Make sure python3 is now pointing to /usr/local/bin/python3 by running which python3.

Then install apache-airflow:

python3 -m venv .venv
pip install apache-airflow
airflow initdb

answered Aug 31 '20 at 19:19

Paul Pawletta

66
4

this issue is micro version-specific, I'm still facing the issue on 3.8.9, but it's working on 3.8.5 - https://github.com/dvarrazzo/py-setproctitle/issues/82 – utkarsh sharma Dec 16 '21 at 13:11
I would recommend using `pyenv` instead of `brew` to manage your python versions. In my experience, `brew` tend to automatically change python version on you (which can totally screw up virtualenvs). `pyenv` lets you actually manage multiple `pythonX.Y.Z` interpreter versions simultaneously. – DeusXMachina Feb 11 '22 at 16:05
`airflow initdb` has been replaced with `airflow db init` – Mehrad Eslami Feb 23 '22 at 04:18

MatrixManAtYrService · Answer 3 · 2021-04-27T22:48:04.630

2

This error is explained more fully in the troubleshooting section here: https://airflow.apache.org/docs/apache-airflow/stable/installation.html

In my case, I had the problem with 3.8.2 installed via homebrew and installing 3.9.4 via pyenv did the trick.

edited Apr 27 '21 at 22:48

answered Apr 27 '21 at 21:50

MatrixManAtYrService

8,023
1
50
61

score 1 · Answer 4 · answered Jul 17 '20 at 15:14

I had the same problem in python 3.7 but Works great for Python 3.8 installed directly from python page, create a new environment and install again apache-airflow in your new environment, be sure you have all dependencies for this version of apache-airflow and you are not using the local environment to avoid errors.

score 1 · Answer 5 · answered Feb 23 '21 at 14:04

1

If you're tied to 3.7.x and using pipenv like me, running the latest patch helps.

brew install pyenv
pyenv install 3.7.10
pyenv global 3.7.10
pipenv --rm
pipenv install --dev --python 3.7.10

answered Feb 23 '21 at 14:04

Tom Rijntjes

614
4
16

Thomas Portwood · Answer 6 · 2020-05-06T22:24:23.057

0

The setproctitle 1.1.10 module works as expected on macOS Catalina when using python 3.8 installed from the official site.

Python 3.8 download: https://www.python.org/downloads/release/python-380/

Related question: Import issue for setproctitle on Mac OS,

edited May 06 '20 at 22:24

answered May 06 '20 at 18:42

Thomas Portwood

1,031
8
13

1

A link to a solution is welcome, but please ensure your answer is useful without it: add context around the link so your fellow users will have some idea what it is and why it’s there, then quote the most relevant part of the page you're linking to in case the target page is unavailable. Answers that are little more than a link may be deleted. – abhiarora May 06 '20 at 18:47
Thanks for your help abhiarora – Thomas Portwood May 06 '20 at 22:25

score 0 · Answer 7 · answered Jun 09 '20 at 15:43

0

I had the same problem with python 3. Try creating an virtualenv of python 2.7 and install airflow in it.

answered Jun 09 '20 at 15:43

SandeepKumar

141
1
1
7

score 0 · Answer 8 · answered Sep 04 '20 at 13:07

0

Python 3.7.3 is causing this issue. I upgraded to Python 3.7.7 and issue got resolved.

mkvirtualenv -p python3.7.7 airflow
workon airflow
pip install apache-airflow
airflow initdb

answered Sep 04 '20 at 13:07

Nitin Bodke

1
1
1

score 0 · Answer 9 · answered Jul 20 '22 at 10:15

I solved the problem by following these steps:

Note: the order of the steps matters!

Install the latest version of python, it was 3.8 now it is 3.9.13.
Create a virtual env.
Update the pip command by executing python3 -m pip install --upgrade pip
Install apache airflow pip install apache-airflow

airflow initdb: undefined symbol: Py_GetArgcArgv

9 Answers9

Linked