1

I have the following code:

readInterface.on('line', async (line) => {
    const splitLine = line.split(',')

    const domain = splitLine[0]
    splitLine.shift()

    const query = `INSERT INTO domains (domain, ips) VALUES(?, ?);`;
    try {
      await client.execute(query, [ domain, splitLine ])
    } catch(e) {
        console.log(e)
    }
});

Currently this doesn't work. The code doesn't await before moving on to the next line. How can I change this to await the execute for each query keeping in mind the file is too big for node and cannot be read into memory?

Quesofat
  • 1,493
  • 2
  • 21
  • 49

1 Answers1

4

If you use the asynchronous iterator of readline, you can, in serial, await each read line and await the database query Promise, so that each goes one-by-one without trying to do all of them all at once:

for await (const line of readInterface) {
    const [domain, ...splitLine] = line.split(',');
    const query = `INSERT INTO domains (domain, ips) VALUES(?, ?);`;
    try {
      await client.execute(query, [ domain, splitLine ])
    } catch(e) {
      console.log(e)
    }
}
CertainPerformance
  • 356,069
  • 52
  • 309
  • 320
  • 1
    Be wary of the `for await ()` use with the `readline` module. It's full of bugs, particularly with error handling situations. I've filed a couple bugs on the module myself and the effort to fix them is very slow. As such, I do not use this construct in my code. In fact, I avoid the whole readline module as it's overloaded to try to do too many things (from tty console prompting to line processing in files) and that is probably partly responsible for the bugs. – jfriend00 Dec 31 '20 at 23:52