Global variable updated in the callback function referenced by the csv-parse module (npm) is not updating

Question

As part of a project with a client, I am developing a workflow on their engine (built partially on top of node.js) and I have a problem with the Javascript task. In particular, I am taking a csv-file accountCsv as an input, then I want to modify a process variable outputFinanceAccounts which should be a string containing some information read from the csv-file. Further, I want to pass this process variable to the next activity of the workflow.

Inside the Javascript task, I am using the module csv.parse (npm package that is available on the client's engine), and in general it works well and reads the data from the csv file correctly. This function uses an async callback function as an argument. Further, I am changing the process variable outputFinanceAccounts inside this async function. During testing I see that this process variable is assigned with correct value inside the parsing block. However, when the csv.parse function is finished working, this updated value of the process variable outputFinanceAccounts is lost. Therefore, I can’t pass any value of this process variable to the next activity of the workflow.

I tried to create a nested function inside csv.parse and return the result of this function outside of csv.parse, but it does not work. From what I understand, this trick would work only if csv.parse would be a function, which it is not really, and I don't know what would be the right syntax for that.

const csv = require('csv');
const files = require('files')

const DepartmentColumn = 'Department'

const options = {auto_parse: true, columns: true, relax_column_count: true }

// Read the reportCsv file (workflow variable)
const csvContents = files.getContent(accountCsv.id)

outputFinanceAccounts = 'VALUE BEFORE PARSING'

console.log('Before parsing', outputFinanceAccounts);

csv.parse(csvContents.buffer.toString('utf-8'), options, (error, AccountList) => {
  if (error) {
      console.log(`Error parsing CSV: ${error}`)
      return
  }
  if (AccountList.length == 0) {
      console.log(`Error: CSV contains zero rows`)
      return
    } else {
      console.log(`${AccountList.length} rows read from CSV`)
  } 
  AccountList.forEach(account => {
   if(account[DepartmentColumn] == 'Finance'){
    outputFinanceAccounts = "VALUE INSIDE PARSING";
    console.log('Inside parsing', outputFinanceAccounts);
   }
  });
})

console.log('After parsing:', outputFinanceAccounts);

Testing the code on the client's workflow engine (built partially using node.js) gives the following results:

Test execution finished successfully

Logs
-> Before parsing VALUE BEFORE PARSING
-> After parsing: VALUE BEFORE PARSING
-> 4 rows read from CSV
-> Inside parsing VALUE INSIDE PARSING
-> Inside parsing VALUE INSIDE PARSING

But what I need to have is:

-> After parsing: VALUE INSIDE PARSING

Thanks in advance for helping me with this.

Possible duplicate of [Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference](https://stackoverflow.com/questions/23667086/why-is-my-variable-unaltered-after-i-modify-it-inside-of-a-function-asynchron) — 1565986223, Sep 03 '19 at 14:17
Not really. The problem is that in my case the async callback function is used as an argument during the call of the csv-parse module . And in the referenced question the problem arise while using "pure" functions. The usage of the external module adds one level of complexity on top of functions. — katberlin, Sep 03 '19 at 14:24
regardless of what you regard as `pure` vs external module, it's a canonical question regarding javascript and async nature. The answer is broad, and has links to various resources — 1565986223, Sep 03 '19 at 14:27
In that sense, thanks for pointing out the valuable source of information. What I meant is that, my question is not a duplicate of the other one. Hope that someone had a similar problem with external modules and knows the syntax how to solve it. — katberlin, Sep 03 '19 at 14:50

1565986223 · Answer 1 · 2019-09-03T15:36:16.417

Your expected output is not possible. console.log('After parsing:', outputFinanceAccounts) always runs before csv.parse(... by the time the variable is updated, console ouput is done.

Just an illustration, but csv.parse is akin to setTimeout here:

let something = "initial"

console.log(something)

setTimeout(() => {
  console.log('this always runs after final console.log')
  something = "final"
}, 0)

// output is still "initial"
console.log(something)

If csv.parse is asynchronous, it's always executed in the future, meaning functions in the stack gets executed first before processing further messages

Here is MDN docs explaining Concurrency and Event Loop Model of JS.

Global variable updated in the callback function referenced by the csv-parse module (npm) is not updating

1 Answers1