2

This seems very simple, but I don't understand ...

use polars::prelude::*;
use std::fs::File;

fn write_df_to_parquet(df: &mut DataFrame) {
    let mut file = File::create("df.parquet").expect("could not create file");

    ParquetWriter::new(&mut file)
    .finish(df)
}

gives use of undeclared type `ParquetWriter` .

Isn't ParquetWriter imported in the use polars::prelude::*; line?

The example given here: https://github.com/pola-rs/polars/blob/51e0e40457d5f2f224df0a4dcf389ced16fbac05/examples/read_csv/src/main.rs

Seems to be accessing ParquetWriter with just that import?

I'm sure I'm missing something silly. Any help or advice would be appreciated.

Progman
  • 16,827
  • 6
  • 33
  • 48
m_fmi
  • 21
  • 1
  • Please add the full error message from `cargo check` (not from an IDE). Yes [`ParquetWriter` should be part of the prelude](https://docs.rs/polars/latest/polars/prelude/struct.ParquetWriter.html) unless you're using a very old version of polars (it was added in v0.12). – Jmb Sep 24 '22 at 17:18

2 Answers2

1

I searched found the implementation of ParquetWriter in the source, and followed the pub use chain. It seams that in the prelude polars-io (the crate in which ParquetWriter is implemented) is listed as a feature.

I suspect the feature you need is not installed. Even the polars-io crate itself has a feature for the parquet structs. It too needs to be installed. To install them add these lines to your Cargo.toml file, in the workspace root.

Note that you need to change the version of polars-io to match to the one of polars, otherwise there will be dependency mismatches.

polars = { version = "0.24.2", features = ["polars-io"] }
polars-io = { version = "0.24.2", features = ["parquet"] }
toni1606
  • 11
  • 1
  • 2
  • Thank you! I had these dependencies in my `Cargo.toml`, but did not have the features called out. Adding them resolved the issue. I am struggling with understanding when to add what "features" to the `Cargo.toml` dependencies. E.g. in the documentation for `ParquetWriter` here: https://docs.rs/polars-io/0.24.1/polars_io/parquet/struct.ParquetWriter.html it says "Available on crate feature `feature` only." Using that did not work, but adding your suggested feature `parquet` did work. I was able to drop the `polars-io` feature with no negative consequences. – m_fmi Sep 26 '22 at 16:18
  • I found the answer to the question you asked [here](https://stackoverflow.com/questions/59761045/how-to-tell-what-features-are-available-per-crate). To sum it up you just go to the [docs.rs](https://docs.rs/) page and go to the features tab. It should list every feature and their uses, but it depends on how good the docs for that particular crate are. – toni1606 Sep 27 '22 at 06:29
  • Thanks @toni1606! This helps go from a crate to what features are defined. But it doesn't help with when I find an element that I want to use, e.g. `ParquetWriter`, but it fails despite adding the dependency to `Cargo.toml`. Here one can guess, but I'm trying to learn about rust. Inspection of the `ParquetWriter` [code](https://github.com/pola-rs/polars/blob/51e0e40457d5f2f224df0a4dcf389ced16fbac05/polars/polars-io/src/parquet/write.rs) doesn't show any hints that a feature is required (e.g. `cfg`? Do you know how can to tell from the code, what feature to include? – m_fmi Sep 27 '22 at 17:30
  • Yes you just go to the definition of the module, not it's implementation in this case it is not in the definition of the `writer` itself, but in it's parent module called `parquet`, as you can see [here](https://github.com/pola-rs/polars/blob/51e0e40457d5f2f224df0a4dcf389ced16fbac05/polars/polars-io/src/lib.rs). You just have to go up the hierarchy of modules until you find the `#[cfg(feature = "")]` on the module in which your type is located. – toni1606 Sep 28 '22 at 16:18
0

Update for later versions

[dependencies.polars]
version = "0.28"
features = ["parquet"]
Edmund's Echo
  • 766
  • 8
  • 15