Error while adding a Dockerfile to install more dependencies

Question

I am using this https://github.com/tuanavu/airflow-tutorial git repo, to run airflow in a docker container, but along with it I want to install additional requirements.

I am trying to install git and few other dependencies for that I have created Dockerfile in the airflow-tutorial directory. and added below lines in the file.

RUN apt-get update && \
    apt-get install -y git

RUN pip install dataclasses

then trying to build it with docker-compose up --build

but it keep failing and error is

   ModuleNotFoundError: No module named 'dataclasses'

it works perfectly fine without my Dockerfile, Can any body please help me solve this issue?

Docker-compose.yml file

version: '3'
services:
  postgres:
    image: postgres:12.3
    environment:
      - POSTGRES_USER=airflow
      - POSTGRES_PASSWORD=airflow
      - POSTGRES_DB=airflow
    ports:
      - "5433:5432"

  webserver:
    image: puckel/docker-airflow:1.10.1
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.1
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: sqlalchemy==1.2.0
    restart: always
    depends_on:
      - postgres
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
      - SSH_AUTH_SOCK=/ssh-agent
    volumes:
      - /home/awaish/work/airflow/dags1:/usr/local/airflow/dags
      - /usr/bin/ssh-agent:/ssh-agent
      # Uncomment to include custom plugins
      # - ./plugins:/usr/local/airflow/plugins
    ports:
      - "8080:8080"
    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

and Dockerfile

#WORKDIR /usr/src/app
FROM python3.7   
RUN apt-get update && \
    apt-get install -y git
RUN pip3 install dataclasses
RUN pip3 install git+ssh://git@bitbucket.org/myspace/utilities.git@master

score 0 · Answer 1 · answered Jun 19 '20 at 13:28

0

If you could provide more information from your error text it would be helpful.

dataclasses is a package available from pypi, so all things considered this should pass. The fallback for pip to error suggesting it cannot find dataclasses is likely that pip can't access pypi or an equivalent repo.

My inkling is that this is probably downstream of docker not being able to contact pypi when it builds your image.

Again, if you'd like to provide more of your error text someone can help.

answered Jun 19 '20 at 13:28

Mike Taylor

689
3
8

Probably the best detail to add is more of the error text from your console. The single line `ModuleNotFoundError: No module named 'dataclasses'`, probably doesn't have all the information people can use to help. – Mike Taylor Jun 22 '20 at 08:52

score 0 · Answer 2 · answered Jun 19 '20 at 13:45

0

Kindly share the complete Dockerfile. If this is the complete file then you need to add more information in the Dockerfile. Read this answer to get complete understanding.

answered Jun 19 '20 at 13:45

aryanveer

628
1
6
17

I have shared a link, it creates an image with alot of requirements but I need to add additional requirements to install so added this Dockerfile in the same directory. but it is not working – Awaish Kumar Jun 19 '20 at 13:52

Error while adding a Dockerfile to install more dependencies

2 Answers2