7

I am using cargo, maturin and pytest to build a mixed Python/Rust project. During development, I frequently cycle through the commands:

$ cargo test -p mypkg --release
$ maturin develop --release
$ python -m pytest --failed-first my_pkg

It seemed like cargo and maturin were compiling dependencies when there was not a reason to do so. After some experimentation, I found that if I run

  • cargo ...
  • maturin ...
  • cargo ...
  • maturin ...

the second run of cargo and maturin would recompile the dependencies, even though I had not manually changed any of the source files.

I don't have a small example to reproduce this, so I am trying to debug it with the full system. To do that, I would like to know what files cargo and/or maturin think are out of date. Once I know that, the full solution will probably be obvious.

However, there don't seem to be flags that I can pass that give me that information. cargo -vv test ... produces a lot of output about what it is compiling and how, but not why. maturin does not even seem to have a -v flag available.

I found cargo-outdated, but that appears to be about dependency versions.

I have two Rust packages, each with 5-10 direct dependencies and about 100 total dependencies.

How can I figure out what files are causing cargo/maturin to rebuild the dependencies?

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
Troy Daniels
  • 3,270
  • 2
  • 25
  • 57

2 Answers2

13

You can ask Cargo to output logging information relevant to fingerprints. With Cargo 1.56.0, the appropriate environment variable is CARGO_LOG=cargo::core::compiler::fingerprint=info.

As an example:

% CARGO_LOG=cargo::core::compiler::fingerprint=info cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 0.00s

% touch src/main.rs
% CARGO_LOG=cargo::core::compiler::fingerprint=info cargo build
[2021-11-30T18:13:54Z INFO  cargo::core::compiler::fingerprint] stale: changed "/private/tmp/xxx/src/main.rs"
[2021-11-30T18:13:54Z INFO  cargo::core::compiler::fingerprint]           (vs) "/private/tmp/xxx/target/debug/.fingerprint/xxx-3af563e7d679143a/dep-bin-xxx"
[2021-11-30T18:13:54Z INFO  cargo::core::compiler::fingerprint]                FileTime { seconds: 1638295984, nanos: 344057437 } != FileTime { seconds: 1638296033, nanos: 750100000 }
[2021-11-30T18:13:54Z INFO  cargo::core::compiler::fingerprint] fingerprint error for xxx v0.1.0 (/private/tmp/xxx)/Build/TargetInner { name: "xxx", doc: true, ..: with_path("/private/tmp/xxx/src/main.rs", Edition2021) }
[2021-11-30T18:13:54Z INFO  cargo::core::compiler::fingerprint]     err: current filesystem status shows we're outdated
Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
3

This is the solution to my specific problem for the next user.

The CARGO_LOG environment variable answer worked. It produced a lot of output along the lines of

[2021-11-30T18:29:06Z INFO  cargo::core::compiler::fingerprint]     err: RUSTFLAGS has changed: previously [], now ["-C", "link-arg=-undefined", "-C", "link-arg=dynamic_lookup"]

I am pretty sure that maturin adds those flags when building a library to be imported by python.

At one point, I saw a way to modify Cargo.toml to add those when building with cargo as well.

My solution was simpler: I ran the cargo tests with

cargo test -p mypkg # no --release

so that cargo builds and tests debug code and maturin/pytest builds and tests release code. The flags are inconsistent, but not in a way that matters. And if maturin changes the flags that it adds, I don't need to update my flags.

This does mean that when dependencies change, I build them twice, but that is much better than building them twice on every cycle.

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366