I wrote an AWS lambda function which that reads a parquet file which has 2 columns whose type is boolean. When I exclude those two boolean columns named: "iscritical" and "iscyclic" from the input columns list the wr.s3.read_parquet() operation success.
code snippet:
valid_cols = [col for col in list(parquet_file_cols_metadata.keys()) if col != "iscritical" and col != "iscyclic"]
stage_file_full_data_df = wr.s3.read _parquet(
path=stage_file,
ignore_empty=True,
use_threads=True,
columns=valid_cols)
When I am trying to read the entire data (inlcude the boolean types columns) the wr.s3.read_parquet() operation fails with exception: "Unknown encoding type"
code snippet:
stage_file_full_data_df = wr.s3.read _parquet(
path=stage_file,
ignore_empty=True,
use_threads=True)
My aim is to read the entire data with the boolean columns.
What I am asking is if someone encountered OR know why wr.s3.read_parquet() cannot handle boolean columns? Thanks.