0

I'm trying to install apache-airflow the recommended way with pip install apache-airflow. During the install of pendulum (a dependency), I get an error:

  error: can't copy 'pendulum/parsing': doesn't exist or not a regular file

I think it's related to Python distutils error: "[directory]... doesn't exist or not a regular file", but that doesn't give an answer as to how one resolves this when using pip. Pulling the tar for pendulum and installing using python setup.py install works, but then when subsequently I do pip install apache-airflow again, it sees that pendulum is already installed, UNINSTALLS, and then tries to install again using pip, resulting in the same error. I'm using a docker container and installing python-setuptools with apt-get before I do any of this. Here's my dockerfile, fwiw...

FROM phusion/baseimage:0.10.1
MAINTAINER a curious dev 

RUN apt-get update && apt-get install -y python-setuptools python-pip python-dev libffi-dev libssl-dev zip wget

ENV SLUGIFY_USES_TEXT_UNIDECODE=yes

RUN wget https://files.pythonhosted.org/packages/5b/57/71fc910edcd937b72aa0ef51c8f5734fbd8c011fa1480fce881433847ec8/pendulum-2.0.4.tar.gz
RUN tar -xzvf pendulum-2.0.4.tar.gz

RUN cd pendulum-2.0.4/ && python setup.py install

RUN pip install apache-airflow

CMD airflow initdb && airflow webserver -p 8080

Does anyone see anything I'm doing wrong? I haven't found anyone else with this error so I think there's something really obvious I'm missing. Thanks for reading.

user1949984
  • 93
  • 1
  • 6

1 Answers1

2

Upgrade pip first.

FROM phusion/baseimage:0.10.1
RUN apt-get update && apt-get install -y python-setuptools python-pip python-dev libffi-dev libssl-dev zip wget
ENV SLUGIFY_USES_TEXT_UNIDECODE=yes
RUN pip install -U pip
RUN pip install apache-airflow
CMD airflow initdb && airflow webserver -p 8080

seems to work fine for me.

AKX
  • 152,115
  • 15
  • 115
  • 172
  • Thanks so much for the help; I appreciate your time! That's odd that apt-get doesn't install the latest version of pip, right? – user1949984 Dec 08 '18 at 20:00
  • System packages usually lag a lot behind reality for Python packages (on Debian, usually ridiculously so; on Ubuntu, slightly less). Either way, `python-pip` installs 8.1.1 and we're at 18.1 now (though the versioning scheme has changed along the way). – AKX Dec 08 '18 at 20:21