1

I am trying to load a parquet file to bigquery using bq comand line utility with option --parquet_enable_list_inference=true. I have confirmed that the number of columns is less than 20 in the file and the row size is not more than 50 mb, maximum it will be 30KB.

When I try to load the same parquet file with option --parquet_enable_list_inference=false load was succesfull. But when the option is enabled it is failing with following error :

Resources exceeded during query execution: UDF out of memory.; Failed to read Parquet file /bigstore/file.parquet. This might happen if the file contains a row that is too large, or if the total size of the pages loaded for the queried columns is too large.

Command used to load :

bq load --replace --source_format=PARQUET --parquet_enable_list_inference=true  ted_analysis.test_table gs://tmp-usage/test.parquet

bq Version - BigQuery CLI 2.0.75

ted
  • 3,911
  • 3
  • 26
  • 49

0 Answers0