Stack Overflow
Stack Overflow

Questions tagged [python-polars]

Polars is a DataFrame library/in-memory query engine.

The Polars core library is written in Rust and uses Arrow, the native arrow2 Rust implementation, as its foundation. It offers Python and JavaScript bindings, which serve as a wrapper for functionality implemented in the core library.

Links

1331 questions
15
votes
1 answer

Polars: Create column with fixed value from variable

I have scrubbed the polars docs and cannot see an example of creating a column with a fixed value from a variable. Here is what works in pandas: df['VERSION'] = version Thx
rchitect-of-info
  • 1,150
  • 1
  • 11
  • 23
15
votes
2 answers

What is the equivalent of `DataFrame.drop_duplicates()` from pandas in polars?

What is the equivalent of drop_duplicates() from pandas in polars? import polars as pl df = pl.DataFrame({"a":[1,1,2], "b":[2,2,3], "c":[1,2,3]}) df Output: shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64…
keiv.fly
  • 3,343
  • 4
  • 26
  • 45
14
votes
1 answer

How to use groupby and apply with polars

I am breaking my head trying to figure out how to use groupby and apply in Python's library polars. Coming from Pandas, I was using: def get_score(df): return spearmanr(df["prediction"], df["target"]).correlation correlations =…
jbssm
  • 6,861
  • 13
  • 54
  • 81
13
votes
1 answer

Py Polars: How to filter using 'in' and 'not in' like in SQL

How can I achieve the equivalents of SQL's IN and NOT IN? I have a list with the required values. Here's the scenario: import pandas as pd import polars as pl exclude_fruit = ["apple", "orange"] df = pl.DataFrame( { "A": [1, 2, 3, 4, 5,…
Daycent
  • 455
  • 4
  • 15
13
votes
1 answer

Easily convert string column to pl.datetime in Polars

Consider a Polars data frame with a column of str type that indicates the date in the format '27 July 2020'. I would like to convert this column to the polars.datetime type, which is distinct from the Python standard datetime. The following code,…
fabioklr
  • 430
  • 1
  • 5
  • 13
12
votes
2 answers

Polars : Is there a json_normalize like feature in Polars?

I went through the entire documentation of Polars but couldn't find anything which could convert nested json into dataframe. test = { "name": "Ravi", "Subjects": { "Maths": 92, "English": 94, "Hindi": 98 } } json_normalize in…
Shikha Sheoran
  • 121
  • 1
  • 4
11
votes
3 answers

Switching between dtypes within a DataFrame

I was trying to search whether there would be a way to change the dtypes for the strings with numbers easily. For example, the problem I face is as follows: df = pl.Dataframe({"foo": ["100CT pen", "pencils 250CT", "what 125CT soever", "this is…
momentlost
  • 111
  • 1
  • 1
  • 3
10
votes
4 answers

In Polars how do I print all elements of a list column?

I have a Polars DataFrame with a list column. I want to control how many elements of a pl.List column are printed. I've tried pl.pl.Config.set_fmt_str_lengths() but this only restricts the number of elements if set to a small value, it doesn't show…
braaannigan
  • 594
  • 4
  • 12
10
votes
2 answers

Polars DataFrame memory size in Python

Was wondering about the size of particular polars DataFrames. I tried with: from sys import getsizeof getsizeof(df) Out[17]: 48 getsizeof(df.to_pandas()) Out[18]: 1602923950 It appears all polars df are 48 bytes? Confused.
fvg
  • 153
  • 3
  • 9
10
votes
5 answers

Pandas REPLACE equivalent in Python Polars

Is there an elegant way how to recode values in polars dataframe. For example 1->0, 2->0, 3->1... in Pandas it is simple like that: df.replace([1,2,3,4,97,98,99],[0,0,1,1,2,2,2])
zenelb
  • 121
  • 2
  • 6
9
votes
1 answer

How to add a column to a polars DataFrame using .with_columns()

I am currently creating a new column in a polars data frame using predictions = [10, 20, 30, 40, 50] df['predictions'] = predictions where predictions is a numpy array or list containing values I computed with another tool. However, polars throws a…
Felix.B
  • 306
  • 2
  • 8
9
votes
2 answers

How to use Polars with Plotly without converting to Pandas?

I would like to replace Pandas with Polars but I was not able to find out how to use Polars with Plotly without converting to Pandas. I wonder if there is a way to completely cut Pandas out of the process. Consider the following test data: import…
fabioklr
  • 430
  • 1
  • 5
  • 13
9
votes
2 answers

How can I append or concatenate two dataframes in python polars?

I see it's possible to append using the series namespace (https://stackoverflow.com/a/70599059/5363883). What I'm wondering is if there is a similar method for appending or concatenating DataFrames. In pandas historically it could be done with…
cnpryer
  • 195
  • 1
  • 1
  • 7
9
votes
1 answer

How to drop row in polars-python

How to add new feature like length of data frame & Drop rows value using indexing. I want to a add a new column where I can count the no-of rows available in a data frame, & using indexing drop rows value. for i in range(len(df)): if…
Hrushi
  • 409
  • 2
  • 10
9
votes
1 answer

Apply function to all columns of a Polars-DataFrame

I know how to apply a function to all columns present in a Pandas-DataFrame. However, I have not figured out yet how to achieve this when using a Polars-DataFrame. I checked the section from the Polars User Guide devoted to this topic, but I have…
Gian Arauz
  • 423
  • 1
  • 7
  • 14
1
2 3
…
88 89