51

I have hello world web project in Rust + Actix-web. I have several problems. First is every change in code causes recompiling whole project including downloading and compiling every crate. I'd like to work like in normal development - it means cache compiled crates and only recompile my codebase. Second problem is it doesn't expose my my app. It's unreachable via web browser

Dockerfile:

FROM rust

WORKDIR /var/www/app

COPY . .

EXPOSE 8080

RUN cargo run

docker-compose.yml:

version: "3"
services:
  app:
    container_name: hello-world
    build: .
    ports:
      - '8080:8080'
    volumes:
      - .:/var/www/app
      - registry:/root/.cargo/registry

volumes:
  registry:
    driver: local

main.rs:

extern crate actix_web;

use actix_web::{web, App, HttpServer, Responder};

fn index() -> impl Responder {
    "Hello world"
}

fn main() -> std::io::Result<()> {
    HttpServer::new(|| App::new().service(web::resource("/").to(index)))
        .bind("0.0.0.0:8080")?
        .run()
}

Cargo.toml:

[package]
name = "hello-world"
version = "0.1.0"
authors = []
edition = "2018"

[dependencies]
actix-web = "1.0"
ckaserer
  • 4,827
  • 3
  • 18
  • 33
Arek C.
  • 1,271
  • 2
  • 9
  • 17

10 Answers10

48

Seems like you are not alone in your endeavor to cache rust dependencies via the docker build process. Here is a great article that helps you along the way: https://blog.mgattozzi.dev/caching-rust-docker-builds/

The gist of it is you need a dummy.rs and your Cargo.toml first, then build it to cache the dependencies and then copy your application source later in order to not invalidate the cache with every build.

Dockerfile

FROM rust
WORKDIR /var/www/app
COPY dummy.rs .
COPY Cargo.toml .
RUN sed -i 's#src/main.rs#dummy.rs#' Cargo.toml
RUN cargo build --release
RUN sed -i 's#dummy.rs#src/main.rs#' Cargo.toml
COPY . .
RUN cargo build --release
CMD ["target/release/app"]

CMD application name "app" is based on what you have specified in your Cargo.toml for your binary.

dummy.rs

fn main() {}

Cargo.toml

[package]
name = "app"
version = "0.1.0"
authors = ["..."]
[[bin]]
name = "app"
path = "src/main.rs"

[dependencies]
actix-web = "1.0.0"

src/main.rs

extern crate actix_web;

use actix_web::{web, App, HttpServer, Responder};

fn index() -> impl Responder {
    "Hello world"
}

fn main() -> std::io::Result<()> {
    HttpServer::new(|| App::new().service(web::resource("/").to(index)))
        .bind("0.0.0.0:8080")?
        .run()
}
ckaserer
  • 4,827
  • 3
  • 18
  • 33
  • 22
    Great answer. For reference, `COPY dummy.rs` can be replaced by `RUN echo "fn main() {}" > dummy.rs`, which avoids having a file `dummy.rs` on the repository. – Jorge Leitao Mar 22 '20 at 07:11
  • 1
    Instead of manipulating the `Cargo.toml` you could add a 2nd `[[bin]]` target to it. Make sure to copy `src/dummy.rs` together with your `Cargo.toml`. Then add a 2nd bin target (like lines 5-7 fro your example). Set `name= "download-only"`, `path = "src/dummy.rs"` and call `cargo build --bin download-only`. – Martin Sommer Mar 14 '21 at 15:59
  • 5
    If you're doing `RUN echo "fn main() {}" > ./src/main.rs`, then `COPY ./src ./src`, cargo may not rebuild your app. You'll need to update the last modified of the main.rs file to inform cargo to rebuild it. Before your 2nd cargo build, you need to run `RUN touch -a -m ./src/main.rs` to update the last modified of the file. – Ari Seyhun Mar 20 '21 at 11:34
  • @MartinSommer the `[[bin]]` approach doesn't work. For your binary `download-only`, you'll need to `cargo build --bin download-only`, which will make docker cache dependencies only for this command. However, you'll still need to `cargo build --bin app`, which won't be cached. The only way for docker to cache properly is if both the build commands are exactly the same, so I don't think there's an alternative to manipulating `Cargo.toml` – a3y3 Apr 22 '21 at 01:44
  • I have found it useful to use the `--offline` flag too to make sure that cargo doesn't attempt to refresh its dependencies as part of the standard build. Though for some reason I'm using `cargo install --offline --path .` instead of `build` but I assume it applies to build too. – MichaelJones May 15 '21 at 10:14
  • 2
    My cargo.toml didn't have a main.rs in it, and incorporating some of the other comments, I came up with this version that does not need sed or dummy.rs. Docker caches by content so I don't think you need any special touch command. `RUN mkdir src` `RUN echo "fn main() {}" > ./src/main.rs` `COPY ["Cargo.toml", "Cargo.lock", "./"]` `RUN cargo build --release` `COPY src src` `RUN cargo build --release` – casret Oct 07 '22 at 23:02
  • @casret I can confirm this works, at least in my case. This is an awesome improvement over other solutions posted here; you might want to post this as a first-class answer to the original question. – lxg Nov 01 '22 at 12:14
  • @Ixg glad that it helped you, I posted it. BTW, I was wrong and the touch is needed for the reason laid out by Ari, I included that in the post. – casret Nov 02 '22 at 16:50
  • 1
    The original blog.mgatozzi.dev article is now 404. The Wayback Machine is your friend: https://web.archive.org/web/20221028051630/https://blog.mgattozzi.dev/caching-rust-docker-builds/ – richb-hanover Jan 07 '23 at 01:19
  • This approach, unfortunately, fails for bigger projects with workspaces. In such a case, cargo chef as explained below is a good option. – Victor Ermolaev Feb 23 '23 at 15:08
  • `RUN touch -t 8001031305 /rust/server/src/main.rs` is also sometimes useful – JBis Jun 10 '23 at 19:43
30

With the (still experimental) Docker Buildkit you can finally properly cache build folders during a docker build step:

Dockerfile:

# syntax=docker/dockerfile:experimental
from rust
ENV HOME=/home/root
WORKDIR $HOME/app
[...]
RUN --mount=type=cache,target=/usr/local/cargo/registry \
    --mount=type=cache,target=/home/root/app/target \
    cargo build --release

Then run:

DOCKER_BUILDKIT=1 docker build . --progress=plain

Subsequent docker builds will reuse the cargo and target folders from cache, hence massively speeding up your builds.

To clear the docker cache mount: docker builder prune --filter type=exec.cachemount

If you don't see proper caching: Make sure to confirm the location of your cargo/registry and target folders in the docker image, if you don't see proper caching.

Minimal working Example: https://github.com/benmarten/sccache-docker-test/tree/no-sccache

electronix384128
  • 6,625
  • 11
  • 45
  • 67
  • Won't using `--mount=type=cache,target=/home/rust/src/target` mean that other simultaneous builds access the `target` directory? Should `sharing=private` or `sharing=locked` be added to that cache line? Perhaps in combination with a separate layer to populate a build local `target/` from the cache and another layer to push the new stuff back into the cache? – Cody Schafer Jan 05 '21 at 23:44
  • If you're using multi-stage build and need copy something from cache (for example built executable binary) you have to copy it out of the cache. For exmaple please see this answer https://stackoverflow.com/a/64141061/4183352 – cymruu Jun 21 '23 at 15:53
13

You can use cargo-chef to leverage Docker layer caching using a multi-stage build.

FROM rust as planner
WORKDIR app
# We only pay the installation cost once, 
# it will be cached from the second build onwards
RUN cargo install cargo-chef 
COPY . .
RUN cargo chef prepare  --recipe-path recipe.json

FROM rust as cacher
WORKDIR app
RUN cargo install cargo-chef
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM rust as builder
WORKDIR app
COPY . .
# Copy over the cached dependencies
COPY --from=cacher /app/target target
RUN cargo build --release --bin app

FROM rust as runtime
WORKDIR app
COPY --from=builder /app/target/release/app /usr/local/bin
ENTRYPOINT ["./usr/local/bin/app"]

It does not require Buildkit and works for both simple projects and workspaces. You can find more details in the release announcement.

LukeMathWalker
  • 131
  • 1
  • 2
  • 1
    Won't using `--mount=type=cache,target=/home/rust/src/target` mean that other simultaneous builds access the `target` directory? Should `sharing=private` or `sharing=locked` be added to that cache line? Perhaps in combination with a separate layer to populate a build local `target/` from the cache and another layer to push the new stuff back into the cache? – Cody Schafer Jan 05 '21 at 23:44
  • I am getting this error: `docker: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "./usr/local/bin/app": stat ./usr/local/bin/app: no such file or directory: unknown.` – Paul Razvan Berg Feb 02 '21 at 13:45
  • 1
    **Update**: I fixed the error by deleting the dot in the array value of "ENTRYPOINT". – Paul Razvan Berg Feb 02 '21 at 13:56
  • This is nice answer. But it lack caching index of crates.io and also it takes long to copy big target directory – S.R Apr 22 '22 at 10:26
6

While electronix384128 answer is excellent. I would like to expand on it by adding cache for .cargo/git which is needed for any dependency using git and by adding a multistage docker example.

Using rust-musl-builder and Docker Buildkit feature, which is now default in Docker Desktop 2.4. On other versions you may still need to enable it via: DOCKER_BUILDKIT=1 docker build .

rusl-musl-builder's working directory is /home/rust/src
Tried setting uid/gid on --mount but failed to compile rust due to permission issue in target.

# syntax=docker/dockerfile:1.2
FROM ekidd/rust-musl-builder:stable AS builder

COPY . .
RUN --mount=type=cache,target=/home/rust/.cargo/git \
    --mount=type=cache,target=/home/rust/.cargo/registry \
    --mount=type=cache,sharing=private,target=/home/rust/src/target \
    sudo chown -R rust: target /home/rust/.cargo && \
    cargo build --release && \
    # Copy executable out of the cache so it is available in the final image.
    cp target/x86_64-unknown-linux-musl/release/my-executable ./my-executable

FROM alpine
COPY --from=builder /home/rust/src/my-executable .
USER 1000
CMD ["./my-executable"]
jetersen
  • 63
  • 3
  • 5
  • Won't using `--mount=type=cache,target=/home/rust/src/target` mean that other simultaneous builds access the `target` directory? Should `sharing=private` or `sharing=locked` be added to that cache line? Perhaps in combination with a separate layer to populate a build local `target/` from the cache and another layer to push the new stuff back into the cache? – Cody Schafer Jan 05 '21 at 23:44
  • 1
    Thanks for the hint, I have updated the example and dockerfile syntax is now no longer experimental – jetersen Feb 07 '21 at 18:03
  • Thank you for the comment you made about copying executable out of the cache. It is very helpful! – cymruu Jun 21 '23 at 15:51
5

Based on @ckaserer's answer, you can RUN echo "fn main() {}" > ./src/main.rs to build the dependencies before building your app.

First copy just your Cargo.toml and Cargo.lock files and build the dummy main.rs file:

FROM rust as rust-builder
WORKDIR /usr/src/app

# Copy Cargo files
COPY ./Cargo.toml .
COPY ./Cargo.lock .

# Create fake main.rs file in src and build
RUN mkdir ./src && echo 'fn main() { println!("Dummy!"); }' > ./src/main.rs
RUN cargo build --release

Then you can copy over your real src directory and run build again:

# Copy source files over
RUN rm -rf ./src
COPY ./src ./src

# The last modified attribute of main.rs needs to be updated manually,
# otherwise cargo won't rebuild it.
RUN touch -a -m ./src/main.rs

RUN cargo build --release

Then we can copy our file to a slim version of debain. Here's the full docker file:

FROM rust as rust-builder
WORKDIR /usr/src/app
COPY ./Cargo.toml .
COPY ./Cargo.lock .
RUN mkdir ./src && echo 'fn main() { println!("Dummy!"); }' > ./src/main.rs
RUN cargo build --release
RUN rm -rf ./src
COPY ./src ./src
RUN touch -a -m ./src/main.rs
RUN cargo build --release

FROM debian:buster-slim
COPY --from=rust-builder /usr/src/app/target/release/app /usr/local/bin/
WORKDIR /usr/local/bin
CMD ["app"]
Ari Seyhun
  • 11,506
  • 16
  • 62
  • 109
3

This is a refinement of the @ckaserer's answer incorporating the comments and some personal experiences. It does not require you create a dummy file in your repo, or edit the Cargo.toml file at build time.

RUN echo 'fn main() { panic!("Dummy Image Called!")}' > ./src/main.rs
COPY ["Cargo.toml", "Cargo.lock",  "./"]
RUN cargo build --release
COPY src src
#need to break the cargo cache
RUN touch ./src/main.rs
RUN cargo build --release
casret
  • 113
  • 1
  • 5
1

I think the problem is your volumes definition is not doing a bind mount. I believe your current config is copying HOST ./registry/ into DOCKER /root/.cargo/registry/, writing to DOCKER /root/.cargo/registry/, and discarding the contents when the container is shut down.

Instead, you'll want to specify the bind type on the volume:

version: "3"
services:
  app:
    container_name: hello-world
    build: .
    environment:
      - CARGO_HOME=/var/www/
    ports:
      - '8080:8080'
    volumes:
      - .:/var/www/app
      - type: bind
        source: ./registry
        target: /root/.cargo/registry

However, keep in mind a /root/.cargo/.package-cache file is also created, but not kept here. Instead, you could change the source to ./.cargo and target to /root/.cargo.


For my own (mostly cli) rust projects, I like to use a drop-in replacement I've written for cargo that I've confirmed caches packages between builds, greatly reducing build time. This can be copied to /usr/local/bin to be used globally, or ran as ./cargo build within a single project. But do keep in mind this specific script assumes the app is located at /usr/src/app within the container, so it'd likely need adjusting for your use.

0b10011
  • 18,397
  • 4
  • 65
  • 86
1

This is what i do, and it's compatible with build scripts. It's a multi-stage build so it results in a small image, but caches the built dependencies in the first image.

FROM rust:1.43 AS builder

RUN apt-get update
RUN cd /tmp && USER=root cargo new --bin <projectname>
WORKDIR /tmp/<projectname>

# cache rust dependencies in docker layer
COPY Cargo.toml Cargo.lock ./
RUN touch build.rs && echo "fn main() {println!(\"cargo:rerun-if-changed=\\\"/tmp/<projectname>/build.rs\\\"\");}" >> build.rs
RUN cargo build --release

# build the real stuff and disable cache via the ADD
ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache
COPY ./build.rs ./build.rs

# force the build.rs script to run by modifying it
RUN echo " " >> build.rs
COPY ./src ./src
RUN cargo build --release

FROM rust:1.43
WORKDIR /bin
COPY --from=builder /tmp/<projectname>/target/release/server /bin/<project binary>
RUN chmod +x ./<project binary>
CMD ./<project binary>
Stephen Eckels
  • 435
  • 6
  • 17
1

I had the exact same problem as you did, and tried multiple ways at shortening the build time by caching dependencies.

1. @ckaserer's answer

It gets the job done, and with the easy-to-understand explanation of why it works, it's a great solution. However, this boils down to preference, but if you don't cache the dependencies this way, you can follow #2.

2. Use cargo-chef

@LukeMathWalker, the creator himself, goes through the steps necessary to use cargo-chef, but here's a *slightly adjusted example from the github page.

Dockerfile

FROM lukemathwalker/cargo-chef:latest-rust-1.60.0 AS chef
WORKDIR /app

FROM chef as planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef as builder
COPY --from=planner /app/recipe.json recipe.json
# Build the dependencies (and add to docker's caching layer)
# This caches the dependency files similar to how @ckaserer's solution
# does, but is handled solely through the `cargo-chef` library.
RUN cargo chef cook --release --recipe-path recipe.json
# Build the application
COPY . .
RUN cargo build --release --bin emailer

FROM debian:buster-slim AS runtime
WORKDIR /app
COPY --from=builder /app/target/release/<Name of Rust Application> /usr/local/bin
ENTRYPOINT ["/usr/local/bin/<Name of Rust Application>"]

You should notice a significant decrease in build times with the above changes!


Side note, as far as I know, this blog entry, though not on dockerized builds, holds the best info on compiling rust apps faster on your local machine. You might find it helpful so I suggest checking it out if you're interested.

DrPoppyseed
  • 31
  • 2
  • 5
0

I realise this answer is a little late to the party, however I believe I have found a solution that is slightly different (although along the same basic idea) but that will create the build dependencies in a single Docker layer, meaning they will be cached - you can simply copy the RUN command layer from the following:

...
COPY Cargo.toml /app/

RUN mkdir src && \
    echo 'fn main() {\nprintln!("Hello, world!");\n}' > src/main.rs && \
    cargo build && \ 
    cargo clean --package $(awk '/name/ {gsub(/"/,""); print $3}' Cargo.toml | sed ':a;N;$!ba;s/\n//g' | tr -d '\r') && \
    rm -rf src 

COPY src /app/src
...

First copy your Cargo.toml file with it's dependencies and then insert the RUN layer below, then afterwards copy in your actual code.

This first runs a dummy application (copied straight from cargo init) and this package is named the same as your project, then it runs cargo clean on just that package, meaning if you add your code and run the build again, all of the dependencies are cached already, and it just rebuilds your code. By combining the whole thing in a single layer it will save time should anything change further down the pipeline.

PJeffes
  • 366
  • 2
  • 11