Polars to_dicts() fails but other methods work

Question

I'm trying to rename columns in a polars DataFrame. Following very useful advice in this question: How to rename column names with first row in polars?

I've attempted to use to_dicts on a dataframe my_df.

my_df.head(1).to_dicts()

This fails with the message:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\me\AppData\Local\Programs\Python\Python39\lib\site-packages\polars\dataframe\frame.py", line 1973, in to_dicts
    return list(self.iter_rows(named=True))
  File "C:\Users\me\AppData\Local\Programs\Python\Python39\lib\site-packages\polars\dataframe\frame.py", line 8383, in iter_rows
    yield from zerocopy_slice.to_arrow().to_pylist()
AttributeError: 'pyarrow.lib.Table' object has no attribute 'to_pylist'
>>> import pyarrow
>>> df_addr_parsed2.head(1).to_dicts()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\me\AppData\Local\Programs\Python\Python39\lib\site-packages\polars\dataframe\frame.py", line 1973, in to_dicts
    return list(self.iter_rows(named=True))
  File "C:\Users\me\AppData\Local\Programs\Python\Python39\lib\site-packages\polars\dataframe\frame.py", line 8383, in iter_rows
    yield from zerocopy_slice.to_arrow().to_pylist()
AttributeError: 'pyarrow.lib.Table' object has no attribute 'to_pylist'

Following this question: How to resolve "ImportError: DLL load failed:" on Python?

I tried using os.add_dll_directory(), which did make a difference. However, it seems that the dll must be loading if other methods are working correctly. I also imported pyarrow.

Has anyone else encountered this problem? I'm using Python 3.9 in Windows, polars 0.18, pyarrow 4.0.1 Any thoughts or advice would be much appreciated.

Can you show us a bit more code, including test data, so we can recreate the issue? If you're having to load a dll, something else seems to be going on. Why do you need `os.add_dll_directory()` at all? — Pep_8_Guardiola, Jun 07 '23 at 13:46
your `my_df` isn't a polars dataframe but a pyarrow table. Do `my_pldf=pl.from_arrow(my_df)` — Dean MacGregor, Jun 07 '23 at 14:10
Does this answer your question? [Create dictionary of each row in polars Dataframe](https://stackoverflow.com/questions/75649804/create-dictionary-of-each-row-in-polars-dataframe) — Pravash Panigrahi, Jun 07 '23 at 14:48

score 0 · Accepted Answer · answered Jun 07 '23 at 14:08

0

It looks like the problem was actually that I had an older version of pyarrow that does not expose the appropriate method: Pyarrow Table doesn't seem to have to_pylist() as a method

Upgrading to pyarrow 12.0 worked.

Thanks all!

answered Jun 07 '23 at 14:08

user13752384

1
3

Polars to_dicts() fails but other methods work

1 Answers1