0

For a single line of code change, I see Heroku installs all the libraries once again, ending up in a longer deployment time, even though I am using Docker. Is there any way to reduce the deployment time?

As per my understanding, Docker uses layers to reduce the build time. It should be applied here as well.

My Dockerfile:

FROM     ubuntu:latest
RUN      apt-get update
RUN      apt-get install -y gcc libffi-dev libssl-dev
RUN      apt-get install -y libxml2-dev xmlsec1
RUN      apt-get install -y python3-pip python3-dev
RUN      pip3 --no-cache-dir install --upgrade pip
RUN      rm -rf /var/lib/apt/lists/*
RUN      mkdir  /app
WORKDIR  /app
COPY     .  /app
RUN      pip3 install -r requirements.txt
# only used if running in local docker container
EXPOSE   5000        
#CMD      /usr/bin/python3  /app/app.py 
ENTRYPOINT  ["/usr/bin/python3", "/app/app.py"]

My heroku.yml

build:
   docker:
      web: Dockerfile
run:
  web: gunicorn app:app

Deployment log

[master 8c586b4] just for debugging; will revert the code changes after fixing
 1 file changed, 1 insertion(+)
/home/ravi/heroku-okta-pysaml2-example>git push heroku master
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 350 bytes | 350.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Compressing source files... done.
remote: Building source:
remote: === Fetching app code
remote: 
remote: === Building web (Dockerfile)
remote: Sending build context to Docker daemon  138.2kB
remote: Step 1/13 : FROM     ubuntu:latest
remote: latest: Pulling from library/ubuntu
remote: 7b1a6ab2e44d: Pulling fs layer
remote: 7b1a6ab2e44d: Verifying Checksum
remote: 7b1a6ab2e44d: Download complete
remote: 7b1a6ab2e44d: Pull complete
remote: Digest: sha256:626ffe58f6e7566e00254b638eb7e0f3b11d4da9675088f4781a50ae288f3322
remote: Status: Downloaded newer image for ubuntu:latest
remote:  ---> ba6acccedd29
remote: Step 2/13 : RUN      apt-get update
remote:  ---> Running in 0bb9b3f1454b
remote: Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
remote: Get:2 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
remote: Get:3 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [733 kB]
remote: Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
remote: Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
remote: Get:6 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
remote: Get:7 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [30.1 kB]
remote: Get:8 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [828 kB]
remote: Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
remote: Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1335 kB]
remote: Get:11 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
remote: Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
remote: Get:13 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [797 kB]
remote: Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1108 kB]
remote: Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1758 kB]
remote: Get:16 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [33.6 kB]
remote: Get:17 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [21.7 kB]
remote: Get:18 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [50.8 kB]
remote: Fetched 20.1 MB in 2s (8365 kB/s)
remote: Reading package lists...
remote: Removing intermediate container 0bb9b3f1454b
remote:  ---> 7685e0782050
remote: Step 3/13 : RUN      apt-get install -y gcc libffi-dev libssl-dev
remote:  ---> Running in 15e80fc6bcfa
remote: Reading package lists...
remote: Building dependency tree...
remote: Reading state information...
remote: The following additional packages will be installed:
remote:   binutils binutils-common binutils-x86-64-linux-gnu cpp cpp-9 gcc-9
remote:   gcc-9-base libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0
remote:   libcrypt-dev libctf-nobfd0 libctf0 libgcc-9-dev libgomp1 libisl22 libitm1
remote:   liblsan0 libmpc3 libmpfr6 libquadmath0 libssl1.1 libtsan0 libubsan1
remote:   linux-libc-dev manpages manpages-dev
remote: Suggested packages:
remote:   binutils-doc cpp-doc gcc-9-locales gcc-multilib make autoconf automake
remote:   libtool flex bison gdb gcc-doc gcc-9-multilib gcc-9-doc glibc-doc libssl-doc
remote:   man-browser
remote: The following NEW packages will be installed:
remote:   binutils binutils-common binutils-x86-64-linux-gnu cpp cpp-9 gcc gcc-9
remote:   gcc-9-base libasan5 libatomic1 libbinutils libc-dev-bin libc6-dev libcc1-0
remote:   libcrypt-dev libctf-nobfd0 libctf0 libffi-dev libgcc-9-dev libgomp1 libisl22
remote:   libitm1 liblsan0 libmpc3 libmpfr6 libquadmath0 libssl-dev libssl1.1 libtsan0
remote:   libubsan1 linux-libc-dev manpages manpages-dev
remote: 0 upgraded, 33 newly installed, 0 to remove and 1 not upgraded.
remote: Need to get 36.1 MB of archives.
remote: After this operation, 152 MB of additional disk space will be used.
remote: Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.10 [1322 kB]
remote: Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 manpages all 5.05-1 [1314 kB]
remote: Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 binutils-common amd64 2.34-6ubuntu1.3 [207 kB]
remote: Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbinutils amd64 2.34-6ubuntu1.3 [474 kB]
remote: Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libctf-nobfd0 amd64 2.34-6ubuntu1.3 [47.4 kB]
remote: Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libctf0 amd64 2.34-6ubuntu1.3 [46.6 kB]
remote: Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 binutils-x86-64-linux-gnu amd64 2.34-6ubuntu1.3 [1613 kB]
remote: Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 binutils amd64 2.34-6ubuntu1.3 [3380 B]
remote: Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gcc-9-base amd64 9.3.0-17ubuntu1~20.04 [19.1 kB]
remote: Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libisl22 amd64 0.22.1-1 [592 kB]
remote: Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 libmpfr6 amd64 4.0.2-1 [240 kB]
remote: Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 libmpc3 amd64 1.1.0-1 [40.8 kB]
remote: Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 cpp-9 amd64 9.3.0-17ubuntu1~20.04 [7494 kB]
remote: Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 cpp amd64 4:9.3.0-1ubuntu2 [27.6 kB]
remote: Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libcc1-0 amd64 10.3.0-1ubuntu1~20.04 [48.8 kB]
remote: Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libgomp1 amd64 10.3.0-1ubuntu1~20.04 [102 kB]
remote: Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libitm1 amd64 10.3.0-1ubuntu1~20.04 [26.2 kB]
remote: Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libatomic1 amd64 10.3.0-1ubuntu1~20.04 [9284 B]
remote: Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libasan5 amd64 9.3.0-17ubuntu1~20.04 [394 kB]
remote: Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 liblsan0 amd64 10.3.0-1ubuntu1~20.04 [835 kB]
Gino Mempin
  • 25,369
  • 29
  • 96
  • 135
myquest1 sh
  • 41
  • 7
  • Instead of `ubuntu:latest`, you could instead pre-build the Docker image with all the dependencies, push that to Dockerhub (or some other image repo Heroku has access to), and then use that as the base image when deploying to Heroku. Building the app image during deployment would just involve `COPY`ing the new/updated app codes. – Gino Mempin Dec 26 '21 at 03:25
  • Do you any suggestion which docker image I have to use instead of ubuntu:latest in this case? I want to run an app with docker, flask, python on Heroku. or give me some pointer how to decide on the image selection – myquest1 sh Dec 26 '21 at 04:06
  • I posted a more complete answer, but what I meant was not to depend on any plain/"vanilla" images like `ubuntu:latest`. Instead build your own base image that includes all the dependencies you need, and then use that for deployment. – Gino Mempin Dec 26 '21 at 11:04
  • When i used ubuntu:latest and installed all dependencies on my local docker then i deployed it on heroku---> this is no more a vanilla image....now when i change my code in my local setup and re-deploy to heroku, it should not rebuild all from scratch. why heroku docker does not keep it in layers? – myquest1 sh Dec 26 '21 at 12:02

1 Answers1

0

As per my understanding, Docker uses layers to reduce the build time. It should be applied here as well.

Docker does cache the results of previous commands (apt-get install, pip3 install, etc) if those command/s did not change (see How to avoid reinstalling packages when building Docker image for Python projects?).

You should see something like this when building locally:

#4 [2/7] RUN apk add --no-cache --update python3 py3-pip bash
#4 sha256:dfd02fb162a43923d5d9ed4c02e3b3134d9b406142e9dab2da1f66a236c32e4d
#4 CACHED

#6 [4/7] RUN pip3 install --no-cache-dir -q -r /tmp/requirements.txt
#6 sha256:fb10329d74bd8ff2ea2c0dd627f6606b2d89824bf44f9c45d9afc5578112e231
#6 CACHED

#8 [6/7] WORKDIR /opt/webapp
#8 sha256:d8fb6d91d96441ed3979461e2e841340a9a8804716dafa52e6dbf54440b2bfef
#8 CACHED

However, this only applies if the Docker context or the build environment used to rebuild the image is the exact same environment used to build the previous version of that image. This cannot be guaranteed when deploying to Heroku as you don't have control over the build or deployment environment. AFAICT, I see no info in Heroku's Building Docker Images docs that allow you to persist the Docker context or build env.

A workaround I can suggest is, if your app dependencies do not change that often (if the apt or Python packages to be installed are almost always the same), is to split your Dockerfile into 2 parts:

  • Dockerfile.base:
    • The base image
    • Has FROM ubuntu:latest + installation of apt and Python packages
  • Dockerfile.app
    • The app image
    • Has FROM <base image> + COPY app codes

The idea is to build and push a base image separately first, then to use that as the FROM value for the Dockerfile of your app's image.

Dockerfile.base

# SAME top part from original Dockerfile
FROM     ubuntu:latest
RUN      apt-get update
RUN      apt-get install -y gcc libffi-dev libssl-dev
RUN      apt-get install -y libxml2-dev xmlsec1
RUN      apt-get install -y python3-pip python3-dev
RUN      pip3 --no-cache-dir install --upgrade pip
RUN      rm -rf /var/lib/apt/lists/*
RUN      mkdir /app
WORKDIR  /app

# --- CHANGED HERE ---
# Only copy the app's requirements.txt
COPY     ./requirements.txt /app/requirements.txt
# Then install the Python packages
RUN      pip3 install -r requirements.txt

Build and push it:
(As an example, I'm naming it myapp and pushing the image to Dockerhub under my-dockerhub-workspace)

$ docker build -t myapp:base -f Dockerfile.base .
$ docker tag myapp:base my-dockerhub-workspace/myapp:base
$ docker push my-dockerhub-workspace/myapp:base

Now you have a Docker image myapp:base that has all the apt and Python dependencies already installed.

Dockerfile.app

# SAME bottom part from original Dockerfile
# BUT removed "pip3 install -r requirements.txt"
FROM     my-dockerhub-workspace/myapp:base
WORKDIR  /app
COPY     .  /app
# only used if running in local docker container
EXPOSE   5000
#CMD      /usr/bin/python3  /app/app.py
ENTRYPOINT  ["/usr/bin/python3", "/app/app.py"]

That simplifies the steps of building the app image, which basically just copies the latest version of your codes into the image. If you just changed a single line of code (that doesn't require any changes with the apt and Python packages), then it wouldn't need to re-install all the dependencies again.

You can test that it builds locally:
(Again, naming it as myapp)

$ docker build -t myapp:1.0.0 -f Dockerfile.app .

So you only need to update your heroku.yml to point to the correct Dockerfile:

build:
   docker:
      web: Dockerfile.app

I should point out that this approach requires 2 considerations.

Consideration #1: Where to store the base image?

I personally use Dockerhub for storing base images. They don't contain any app codes and they rarely need to be updated, so it's fine even if I store them under Dockerhub's public repositories. And Heroku can pull from Dockerhub repositories just fine.

I read that Heroku has its own Container Registry which requires you to prefix your image names with registry.heroku.com:

$ docker tag <image> registry.heroku.com/<app>/<process-type>
$ docker push registry.heroku.com/<app>/<process-type>

The idea seems to be the same, AFAICT, from the Heroku docs on:

Unfortunately, I haven't personally tried it and I read from these posts that it doesn't work as a storage for base images:

One of the OP self-answered with an almost similar solution:

I decided to create a series of base images for our different language stacks, host those on Dockerhub, and use those as a starting point for my review apps' images.

Consideration #2: How to version the base image?

You will have to setup some sort of workflow on when do you build and push and version your base image. It has to be in-sync with what your app image needs. For example, if you are using Github to store your app codes, you might want to setup a Github Action that pushes a new base image every time the requirements.txt file changes on a specific branch, such as the master/main branch.

Gino Mempin
  • 25,369
  • 29
  • 96
  • 135