1

This question is different from the one linked as already having an answer. It is specifically this piece of code adapted from node.js documentation regarding the use of fs and readfile and looking for an end of file flag, which I've learned is readfile.close method. Thanks for the answer.

I wrote a small utility locally to try to convert a text file of key:value pairs with a blank line separating programs into a JSON file to use in a React project.

I got the foundation of the readline function directly from node.js documentation. I'm using node 6.9.0 on my mac

here is the full script: 

const readline = require('readline');
const fs = require('fs');

const rl = readline.createInterface({
  input: fs.createReadStream('usat-ncaa-programs.txt')
});

var newPairs=["test"];
rl.on('line',
  function (line) {
  if (line===null){
    newPairs.push("}], [ {")
  } else if (line) {
    var keyValue = line.match(/^(.*):(.*)/)
    var newKeyValuePair =  "'" + keyValue[1].trim() + "':  '" + keyValue[2].trim() + "'"
    newPairs.push(newKeyValuePair)
    //console.log (newKeyValuePair)
  }

})

console.log(newPairs)

The input file looks like this (there are about 12 programs), i've only included 2 1/2 so you can see the format: 

University: Arizona State University
Division: I
University Home Page: http://www.asu.edu/
Recruiting Link: https://questionnaire.acsathletics.com/Questionnaire/Questionnaire.aspx?&SPSID=1061112&SPID=177408&DB_LANG=C&DB_OEM_ID=30300&q=2015&s=159130&o=143
Team Homepage: http://www.thesundevils.com/index.aspx?path=triathlon
Head Coach: Cliff English
w: 480.965.0546
e: cliff.endlish@asu.edu
bg-color: #990033
color: #FFB310

University: Belmont Abby College
Division: II
University Home Page: http://belmontabbeycollege.edu/
Recruiting Link: https://coach.scoutforce.com/p/414f3219dd
Team Homepage: http://abbeyathletics.com/sports/wtri/index
Head Coach: Nick Radkewich
w: 704.461.5010
e: NicholasRadewich@bac.edu
Twitter: https://twitter.com/AbbeyTri
bg-color: #FFFDD0
color: #DC143C

University:Black Hills State University 
Division: II
University Home Page: http://www.bhsu.edu/
...

My problem is that while I can read the text file line by line and parse some information that looks like a JSON file, I am unable to access that data outside of the callback function.

I don't know how to save this data into a new file, or even just output the object to my console for a cut & paste and manually edit.

In teh above script the logged output of the variable newPairs is ["test"] rather than the line by line parsing that I'm trying to acccomplish.

If I place the console.log inside the callback, I get information logged with every iteration of the file read. I only want to work with the data when the file is done.

I did not find an EOF or similar flag in the node documentation for either fs or readline.

Additionally, if there is an easier way to get the data I am inputting into JSON format, I'd love to hear. Thanks in advance.

Shazam
  • 301
  • 1
  • 2
  • 16
  • You should read [Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference](http://stackoverflow.com/questions/23667086/why-is-my-variable-unaltered-after-i-modify-it-inside-of-a-function-asynchron) – Felix Kling Jan 10 '17 at 06:08
  • thanks I did take a look. I knew it had to do with being asynchronous but was not experienced with the readfile methods. Thanks again, I appreciate your help. – Shazam Jan 10 '17 at 06:11
  • I can only recommend to always take a look at the complete documentation. In this case: https://nodejs.org/api/readline.html#readline_event_close . You probably also want to read a bit more about streams: https://github.com/substack/stream-handbook – Felix Kling Jan 10 '17 at 06:11
  • I can definitely learn more about streams. I did read the full documentation on readline and for whatever reason, likely because this is new to me, it did not jump out as the answer or solution to this issue. I was looking for something specifically in the 'on' method indicating an end of file or stream. Thanks for the additional suggestions about streams. – Shazam Jan 10 '17 at 06:17
  • 1
    Sure, you can't know something if you don't know it ;) Most Node APIs that are event emitters / expose a function to bind events (i.e. `.on`) have a section about what events they emit and what those events do. Now you know :) Happy coding! – Felix Kling Jan 10 '17 at 06:22

2 Answers2

3

You have to understand that the callback function is executed asynchronously. This means that console.log(newPairs) gets executed before your callback, therefore it only results in "test".

You should listen to Readline's close event, like so:

rl.on('close', function() {
  console.log(newPairs);
});

As the documentation states:

The 'close' event is emitted when one of the following occur:

  • The rl.close() method is called and the readline.Interface instance has relinquished control over the input and output streams;
  • The input stream receives its 'end' event; The input stream receives -D to signal end-of-transmission (EOT);
  • The input stream receives -C to signal SIGINT and there is no SIGINT event listener registered on the readline.Interface instance.
  • The listener function is called without passing any arguments.

The readline.Interface instance should be considered to be "finished" once the 'close' event is emitted.

So this would be the 'EOF' you're looking for :-)

sgtdck
  • 1,008
  • 7
  • 15
  • I know it's asynchronous but I didn't know how to work with it to get the full file read. I'll try the close event and see if that does the trick. I was looking for an "end of file" sort of method. This worked and I have marked it as the answer. Thank you for your help. – Shazam Jan 10 '17 at 06:01
  • I've updated the answer with an excerpt from the doco – sgtdck Jan 10 '17 at 06:04
  • well it won't let me accept the answer yet, and I'm headed to bed for a fresh start...will accept as answer in the morning. Thanks again. – Shazam Jan 10 '17 at 06:06
  • No worries. Good luck! – sgtdck Jan 10 '17 at 06:07
1

i know its an old question but the answer didn't help me. You can use an promisse like this:

function readLineAsync(){
 var data = [];
 var i = 0;
 return new Promise((resolve,reject)=>{
     var lineRead = readLine.createInterface({
         input:fs.createReadStream('path.to.file')
     })
     lineRead.on('line',(line)=>{
        data[i] = line;
        i++; 
     })
     lineRead.on('close',()=>{
        resolve(dataCsv);
     })
 })
}
(async function(){ console.log(await readLineAsync())}())
JsHitachi
  • 69
  • 3
  • 8