How can I update a parquet file in typescript?

Question

I'd like to append records to an existing parquet file with parquetjs and typescript. Here is the code I already writen:

import ParquetWriterOpts, { ParquetWriter, ParquetReader } from 'parquetjs';
import { ParquetSchema } from 'parquetjs/lib/schema';


var schema = new ParquetSchema({
    name: { type: 'UTF8' },
    quantity: { type: 'INT64' },
    price: { type: 'DOUBLE' },
    date: { type: 'TIMESTAMP_MILLIS' },
    timezone: { type: 'INT_8' },
    in_stock: { type: 'BOOLEAN' }
});

const test = async () => {

    const writer = await ParquetWriter.openFile(schema, 'fruits.parquet');

    let recordedAt = new Date();
    await writer.appendRow({ name: 'apples', quantity: 10, price: 2.5, date: recordedAt, timezone: recordedAt.getTimezoneOffset(), in_stock: true });
    await writer.appendRow({ name: 'oranges', quantity: 10, price: 2.5, date: recordedAt, timezone: recordedAt.getTimezoneOffset(), in_stock: true });
    await writer.close();

    let reader = await ParquetReader.openFile('fruits.parquet');

    let cursor = reader.getCursor();

    let record = null;
    while (record = await cursor.next()) {
        console.log(record);
    }

    reader.close()
}

test();

File is overwiten by this code instead append new record. How can I append data to an existing records in parquetjs?

How can I update a parquet file in typescript?

0 Answers0