Questions tagged [parquetjs]
8 questions
2
votes
2 answers
How can I figure out why BigQuery is rejecting my parquet file?
When trying to upload a parquet file into BigQuery, I get this error:
Error while reading data, error message: Read less values than expected from: prod-scotty-45ecd3eb-e041-450c-bac8-3360a39b6c36; Actual: 0, Expected: 10
I don't know why I get…
Dobes Vandermeer
- 8,463
- 5
- 43
- 46
1
vote
0 answers
AWS Athena error when querying nested schema stored as Parquet format
I'm trying to query data from S3 using AWS Athena, where the data is stored in Parquet format. Specifically, I am trying to create a nested schema that stores rows of a complex object, generated using the parquetjs library. Here is an example of how…
Caesar
- 9,483
- 8
- 40
- 66
1
vote
3 answers
Converting JSON to Parquet in a NodeJS lambda to write into S3
I am running an AWS Lambda function with NodeJS as the language. This lambda receives some JSON input that I need to transform into Parquet format before writing it to S3.
Currently, I'm using the parquetjs library to convert the JSON data to…
Vikram Sodhan
- 21
- 5
1
vote
1 answer
Parquet write to gcs is not queryable by bigquery in nodejs
i'm using parquetjs to create parquet files and push to google cloud storage.
Problem is that bigquery cannot read the data from file but when i use parquet-tools everything looks healthy.
Mahan Hazrati Sagharchi
- 190
- 13
1
vote
0 answers
cursor.buffer.readInt32LE is not a function
I use NuxtJS and I would like to read a parquet file from a URL. So I try to use the parquetjs-lite library (https://github.com/ZJONSSON/parquetjs). However, I get an error when I read the records using the cursor.next() method. The file seems to be…
Rod
- 712
- 2
- 12
- 36
0
votes
0 answers
How can I update a parquet file in typescript?
I'd like to append records to an existing parquet file with parquetjs and typescript. Here is the code I already writen:
import ParquetWriterOpts, { ParquetWriter, ParquetReader } from 'parquetjs';
import { ParquetSchema } from…
Papp Zoltán
- 131
- 7
0
votes
1 answer
How to upload parquet file to s3 bucket in nodejs?
I am using parquetjs library. It is allowing me to create the parquet file, and save it locally, but I want to send it directly to the s3 bucket after creating it. The online examples did not help really. I am putting the code I am using…
Aijaz
- 680
- 15
- 42
0
votes
1 answer
Reading large CSV file on a JavaScript frontend application
I have several CSV files that I'd like to use in a JavaScript based front-end application. Most of them are stored on a Cloud. Considering the fact that some of the CSV are sometimes very big (several gigabytes), I first considered using parquetjs…
accpnt
- 91
- 10