node.js - loop through unnamed objects

Question

I have a directory of files that a service outputs and each file has no extension. Example file names:

all_events_20170406v1
all_events_20170406v2

Within each file are several unnamed JSON objects such as:

{"event":"event1","id":"123"}
{"event":"event2","id":"456","test":"text","foo":"bar"}

Using node.js I'd like to loop through each file, and then within each object of the file, then capture the key names for deduplication. I can't quite figure out how to read the unnamed objects.

My needed output would be:

event
id
test
foo

Any Suggestions?

read the file one line at a time, parsing each line and appending to an array (or processing on the spot) node.js docs has an example. — Kevin B, Apr 06 '17 at 21:17

score 0 · Answer 1 · edited May 23 '17 at 11:54

Ok, use glob module (npm install glob):

let result = []; 
const files = glob.sync('*', { cwd: 'my_folder' });
for (const filename of files) {
    const filePath = 'my_folder/' + filename;
    const content = fs.readFileSync(filePath, 'utf8');
    const lines = content.split('\n');
    const objects = lines.map(line => JSON.parse(line));

    for (const object of objects) {
        for (const key in object) {   // pay attention that "in" used here
            if (Object.hasOwnProperty(key) && result.indexOf(key) === -1) result.push(key);
        }
    }
}
console.dir(result);

Disclaimer: this code is not tested. Provided just for reference. Also, here we read whole file at once, which is ok for small-mid files. If you work with larger files, then use readline module. Please refer to Read a file one line at a time in node.js? for farther details.

node.js - loop through unnamed objects

1 Answers1