1

I want to open all parquet files existing in a specific folder and draw a scatter plot with it. I used following code: 

for file in glob.glob("*.parquet"):
    with pd.read_parquet(file, columns=["cordx", "cordy"]) as df:
        make some scaterplot

Files can be found as I let python print the file sucessfully. But i recive AttributeError: __enter__. And read_parquet is the corredct way to open those files. When I dont use columns=[ ] I get pyarrow.lib.ArrowIOError: Arrow error: Out of memory: malloc of size 9771487328 failed error because the files are very big. So its necessary to load only those columns.

Amir
  • 399
  • 1
  • 7
  • 25

1 Answers1

3

This error tells you that pandas.read_parquet does not have a context manager implemented which means that you cannot call it using with because __enter__ and __exit__ methods are missing. Look in here How to read a Parquet file into Pandas DataFrame?

Colonder
  • 1,556
  • 3
  • 20
  • 40