Read json file content with require vs fs.readFile

Question

Suppose that for every response from an API, i need to map the value from the response to an existing json file in my web application and display the value from the json. What are the better approach in this case to read the json file? require or fs.readfile. Note that there might be thousands of request comes in at a same time.

Note that I do not expect there is any changes to the file during runtime.

request(options, function(error, response, body) {
   // compare response identifier value with json file in node
   // if identifier value exist in the json file
   // return the corresponding value in json file instead
});

Answer 1

I suppose you'll JSON.parse the json file for the comparison, in that case, require is better because it'll parse the file right away and it's sync:

var obj = require('./myjson'); // no need to add the .json extension

If you have thousands of request using that file, require it once outside your request handler and that's it:

var myObj = require('./myjson');
request(options, function(error, response, body) {
   // myObj is accessible here and is a nice JavaScript object
   var value = myObj.someValue;

   // compare response identifier value with json file in node
   // if identifier value exist in the json file
   // return the corresponding value in json file instead
});

Answer 2

There are two versions for fs.readFile, and they are

Asynchronous version

require('fs').readFile('path/test.json', 'utf8', function (err, data) {
    if (err) 
       // error handling

    var obj = JSON.parse(data);
});

Synchronous version

var json = JSON.parse(require('fs').readFileSync('path/test.json', 'utf8'));

---

To use require to parse json file as below

var json = require('path/test.json');

But, note that

• require is synchronous and only reads the file once, following calls return the result from cache

• If your file does not have a .json extension, require will not treat the contents of the file as JSON.

Answer 3

Since no one ever cared to write a benchmark, and I had a feeling that require works faster, I made one myself.

I compared fs.readFile (promisified version) vs require (without cache) vs fs.readFileSync.

You can see benchmark here and results here.

For 1000 iterations, it looks like this:

require: 835.308ms
readFileSync: 666.151ms
readFileAsync: 1178.361ms

So what should you use? The answer is not so simple.

1. Use require when you need to cache object forever. And better use Object.freeze to avoid mutating it in application.
2. Use readFileSync in unit tests or on blocking application startup - it is fastest.
3. Use readFile or promisified version when application is running and you don't wanna block event loop.

Answer 4

Use node-fixtures if dealing with JSON fixtures in your tests.

The project will look for a directory named fixtures which must be child of your test directory in order to load all the fixtures (*.js or *.json files):

// test/fixtures/users.json
{
  "dearwish": {
    "name": "David",
    "gender": "male"
  },
  "innaro": {
    "name": "Inna",
    "gender": "female"
  }
}

// test/users.test.js
var fx = require('node-fixtures');
fx.users.dearwish.name; // => "David" 

Answer 5

I only want to point out that it seems require keeps the file in memory even when the variables should be deleted. I had following case:

for (const file of fs.readdirSync('dir/contains/jsons')) {
  // this variable should be deleted after each loop
  // but actually not, perhaps because of "require"
  // it leads to "heap out of memory" error
  const json = require('dir/contains/jsons/' + file);
}

for (const file of fs.readdirSync('dir/contains/jsons')) {
  // this one with "readFileSync" works well
  const json = JSON.parse(fs.readFileSync('dir/contains/jsons/' + file));
}

The first loop with require can't read all JSON files because of "heap out of memory" error. The second loop with readFile works.

Answer 6

{
  "country": [    
    "INDIA",
    "USA"
  ],
  "codes": [   
    "IN",
    "US"
  ]
}

// countryInfo.json

const { country, code } = require('./countryInfo.json');

console.log(country[0]); // "INDIA"
console.log(code[0]); // "IN"

Answer 7

If your file is empty, require will break. It will throw an error:

SyntaxError ... Unexpected end of JSON input.

With readFileSync/readFile you can deal with this:

let routesJson = JSON.parse(fs.readFileSync('./routes.json', 'UTF8') || '{}');

or:

let routesJson
fs.readFile('./dueNfe_routes.json', 'UTF8', (err, data) => {
    routesJson = JSON.parse(data || '{}');
});