cursor.buffer.readInt32LE is not a function

Question

I use NuxtJS and I would like to read a parquet file from a URL. So I try to use the parquetjs-lite library (https://github.com/ZJONSSON/parquetjs). However, I get an error when I read the records using the cursor.next() method. The file seems to be of the right format because I can download it via the url and read it via this website: https://parquet-viewer-online.com

I forgot a dependency?

My code:

async readParquetFile(url) {
    const parquet = require('parquetjs-lite');
    const request = require('request');
    let reader = await parquet.ParquetReader.openUrl(request, url)
    let cursor = reader.getCursor();
    let record = null;
    while (record = await cursor.next()) {   // error TypeError: cursor.buffer.readInt32LE is not a function
        console.log(record);
    }
}

Error:

TypeError: cursor.buffer.readInt32LE is not a function
    at decodeValues_INT32 (plain.js?8f9a:42)
    at Object.exports.decodeValues (plain.js?8f9a:251)
    at decodeValues (reader.js?245a:614)
    at decodeDataPage (reader.js?245a:832)
    at decodePage (reader.js?245a:670)
    at decodePages (reader.js?245a:713)
    at eval (reader.js?245a:577)
    at async ParquetEnvelopeReader.readRowGroup (reader.js?245a:535)
    at async ParquetCursor.next (reader.js?245a:60)

I use NuxtJs with these dependencies in package.json

"dependencies": {
    "@nuxtjs/axios": "latest",
    "@nuxtjs/pwa": "latest",
    "fs": "latest",
    "net": "latest",
    "tls": "latest",
    "nuxt": "latest",
    "parquetjs-lite": "latest",
    "request": "latest"
},

and with this build in nuxt.config.js

build: {
    extend(config, {}) {
        config.node = {
            fs: 'empty',
        }
    }
},

Shouldn't it be `while (record === await cursor.next()) {` rather than an affectation ? Be careful with using `latest` in your `package.json` btw, can be risky. Also, a quick recommendation, use `import parquet from 'parquetjs-lite'` since you're using modern code. — kissu, May 04 '21 at 14:15
Thanks your for your tips tips on the `import` . However, the code for the `while` is from the tutorial: https://github.com/ZJONSSON/parquetjs#usage-reading-files .. — Rod, May 04 '21 at 14:41

cursor.buffer.readInt32LE is not a function

0 Answers0