Can I prevent cargo from rebuilding libraries with every new project?

Question

Suppose I execute cargo new one --bin and cargo new two --bin then add the same dependency to each project's Cargo.toml and build them.

Now there are two absolutely identical sets of libraries:

/one/target/debug/deps/ *.rlib

/two/target/debug/deps/ *.rlib

They are same files and waste storage space, but really the problem is that I have to compile these libraries again for every project. It takes a very much time. There is the same problem with cargo install.

Can I specify a place to store compiled libraries to avoid recompilation?

Answer 1

Several Cargo projects might share the libraries by using the same target dir.

.cargo/config

Place a ".cargo" folder in a project and create a "config" file there containing:

[build]
target-dir = "/path/to/your/shared/target/dir"

On Unix this might look like:

mkdir ~/shared_rust_target
mkdir .cargo
echo "[build]" > .cargo/config
echo "target-dir = \"$HOME/shared_rust_target\"" >> .cargo/config

CARGO_TARGET_DIR

Set the CARGO_TARGET_DIR environment variable.

On Unix this might look like:

export CARGO_TARGET_DIR="$HOME/shared_rust_target"

---

See this commit for some extra target-dir documentation.

In particular, prior to Cargo 1.9 you shouldn't build several projects into the same target dir concurrently. (Here's more on how Cargo 1.9 supports concurrent builds).

target-dir is also mentioned in the Cargo docs.

Should work, according to this issue.

---

P.S. It is now also possible to achieve crate reuse with workspaces.

---

P.S. https://docs.rs/cargo-hakari/latest/cargo_hakari/ helps with keeping some of dependencies compatible between projects.

Answer 2

Even if there is a way to do it, you probably don't want to. Just because you happen to be using the same libraries doesn't mean that they were compiled the same. For example, Cargo supports the concept of features, compilation time configuration that changes how the crate was compiled.

Likewise, you may need to support multiple versions of Rust, such as nightly and stable. Or perhaps you need to cross-compile for a different architecture. Each of those will produce different code.

Cargo will cache the build products of a single project, so I've found the overhead not really noticeable, and I compile a lot of projects from people asking questions on Stack Overflow! :-)

Answer 3

I managed to piece together code from a few answers, but mainly this one. This Dockerfile should not just cache Cargo dependancy downloads, but also their compilations and the crates.io index. All the other answers I could find only cached the downloads or the index, not both.

FROM arm64v8/rust as builder

# Capture dependencies
COPY Cargo.toml Cargo.lock /app/

# We create a dummy main.rs to build deps
WORKDIR /app
RUN mkdir src && echo "fn main() {}" > src/main.rs
# RUN rustup install nightly && rustup default nightly

# This step compiles only our dependencies and saves them in a layer. This is the most impactful time savings
# Note the use of --mount=type=cache. On subsequent runs, we'll have the crates already downloaded
RUN --mount=type=cache,target=/usr/local/cargo/registry cargo build --release && rm src/main.rs

# Copy our sources
COPY ./src /app/src

# A bit of magic here!
# * We're mounting that cache again to use during the build, otherwise it's not present and we'll have to download those again - bad!
# * Rust here is a bit fiddly, so we'll touch the files (even though we copied over them) to force a new build
RUN --mount=type=cache,target=/usr/local/cargo/registry \
    set -e && \
    # update timestamps to force a new build &&
    touch /app/src/main.rs && \
    cargo build --release

# Again, our final image is the same - a slim base and just our app
FROM debian:buster-slim as app
COPY --from=builder /app/target/release/app/app
CMD ["/app"]

Take note of the FROM arm64v8, if you are targeting x86, replace the builder and app FROMs with their respective x86 versions.