1

i'm using parquetjs to create parquet files and push to google cloud storage.

Problem is that bigquery cannot read the data from file but when i use parquet-tools everything looks healthy.

Mahan Hazrati Sagharchi
  • 190
  • 13

1 Answers1

3

just pass useDataPageV2: false as option to parquet.ParquetWriter.openFile(...)

Like this:

const writer = await parquet.ParquetWriter.openFile(schema, filename, { useDataPageV2: false })
Mahan Hazrati Sagharchi
  • 190
  • 13
  • I had to search in github's repo to find it. incase you are stuck and using aws/s3/athena you can use the same solution. for more info visit: https://github.com/ironSource/parquetjs/issues/124 – Mahan Hazrati Sagharchi Nov 23 '21 at 15:48