838

Background

I am doing some experimentation with Node.js and would like to read a JSON object, either from a text file or a .js file (which is better??) into memory so that I can access that object quickly from code. I realize that there are things like Mongo, Alfred, etc out there, but that is not what I need right now.

Question

How do I read a JSON object out of a text or js file and into server memory using JavaScript/Node?

Community
  • 1
  • 1
Matt Cashatt
  • 23,490
  • 28
  • 78
  • 111

13 Answers13

1519

Sync:

var fs = require('fs');
var obj = JSON.parse(fs.readFileSync('file', 'utf8'));

Async:

var fs = require('fs');
var obj;
fs.readFile('file', 'utf8', function (err, data) {
  if (err) throw err;
  obj = JSON.parse(data);
});
mihai
  • 37,072
  • 9
  • 60
  • 86
  • 22
    I think JSON.parse is synchronous, its directly from v8, which means even with the Async way, people have to be careful with large JSON files. since it would tie up node. – Sean_A91 Aug 03 '15 at 04:29
  • 30
    For the sake of completeness. Their exists a npm called [jsonfile](https://www.npmjs.com/package/jsonfile). – Stefan Feb 23 '16 at 15:37
  • what's wrong with this ? @mihai [Async-json-error](https://raw.githubusercontent.com/xgqfrms-GitHub/webgeeker/gh-pages/JSON/images/Async-json-error.png) [gist code](https://gist.github.com/xgqfrms-GitHub/c1b34a760db12ed50475e3733e6e1c58) – xgqfrms Nov 02 '16 at 21:02
  • 7
    I cant believe it was so difficult to find this simple thing. Every answer I got from google was doing an HTTPRequest or using JQuery or doing it in the browser – juliangonzalez Jun 14 '17 at 17:26
  • 8
    two points: (1) The synchronous answer should just be `let imported = require("file.json")`. (2) JSON.parse must be asynchronous, because I used this code to load a 70mb JSON file into memory as an object. It takes milliseconds this way, but if I use `require()`, it _chugs_. – Kyle Baker Apr 03 '18 at 04:09
  • 2
    One more point: this is async, but it isn't _streaming_. For that, you'll need a library (e.g., Oboe.js). The difference is async won't block your execution thread, but unless you're streaming you'll still see massive spikes in memory consumption (like 450mb of memory to process a 70mb JSON file). – Kyle Baker Apr 04 '18 at 20:36
  • 3
    @KyleBaker `JSON.parse` is synchronous. `require` just does a whole lot more than `JSON.parse`. `JSON.parse` is just parsing JSON. `require` is parsing JavaScript. I'm sure you can figure out why the latter is slower. – Nic Oct 31 '18 at 07:11
  • Great, Thank You :) i was trying with require("file.json") which was working fine but my file.json 's path was not fix for every os so it was failing now it works great. thanks – agravat.in Jan 16 '19 at 04:43
  • 43
    For people finding this answer in 2019 and on, Node.js has had native json support for many, many versions through `require`, with this answer is no longer being applicable if you just want to load a json file. Just use `let data = require('./yourjsonfile.json')` and off you go (with the booknote that if the performance of require is impacting your code, you have problems well beyond "wanting to load a .json file") – Mike 'Pomax' Kamermans Feb 11 '19 at 03:12
  • The asynchronous example is a bit deceiving, since you store the data in `obj`, which is not available for the rest of the running synchronous code. See: [How do I return the response from an asynchronous call?](https://stackoverflow.com/questions/14220321/how-do-i-return-the-response-from-an-asynchronous-call) The better option for the asynchronous example might be `var fs = require("fs").promises` then `var pObj = fs.readFile("file").then(JSON.parse)` where `pObj` is a promise containing the parsed JSON data when resolved. – 3limin4t0r Apr 09 '20 at 10:59
  • This returns the buffer and not the file contents. does not answer the questionn – Luke Pring Jan 13 '21 at 16:03
  • Would `JSON.Parse` have the advantage that it's not possible to inject actual javascript, whereas `require` does allow that? – Nick.Mc Jun 01 '23 at 05:13
479

The easiest way I have found to do this is to just use require and the path to your JSON file.

For example, suppose you have the following JSON file.

test.json

{
  "firstName": "Joe",
  "lastName": "Smith"
}

You can then easily load this in your node.js application using require

var config = require('./test.json');
console.log(config.firstName + ' ' + config.lastName);
Travis Tidwell
  • 5,360
  • 1
  • 14
  • 9
  • 41
    Just so folks know, and if I remember correctly, `require` in node runs synchronously. Dive in deep [here](http://fredkschott.com/post/2014/06/require-and-the-module-system/) – prasanthv Nov 05 '14 at 03:04
  • @Ram It works for me may be your location of `test.json` would be different from his example. I am using `node 5.0` – guleria Nov 30 '15 at 03:40
  • 27
    Another issue/benefit with such method is the fact that required data is cached unless you specifically delete the cached instance – magicode118 Jan 01 '16 at 11:46
  • 22
    "require" is meant to be used to load modules or config file you are using through out the lifespan of your application. does not seem right to use this to load files. – Yaki Klein May 18 '16 at 09:55
  • 5
    I'd say this is potentially a security threat. If the json file you're loading contains JS code, would `require`ing it run that code? If so then you really need to control where your json files are coming from or an attacker could run malicious code on your machine. – sokkyoku Apr 04 '19 at 17:46
  • 7
    This is a sound solution for small DevOps scripts or batch operations. You have to balance human time with performance. As far as something you can commit to memory and use quickly for these appropriate cases, this is is tops. Not every task involves Big Data™ and hostile execution environments. – Bruno Bronosky Mar 20 '20 at 15:25
  • 1
    From the documentation, it looks like file with json extension will be parsed as text files, so would it still be secruity issue. https://nodejs.org/api/modules.html#modules_file_modules – gvmani Apr 23 '20 at 12:13
  • [here's another good article](https://blog.codingblocks.com/2018/reading-json-files-in-nodejs-require-vs-fs-readfile/) to help you make an informed decision – dragon Apr 26 '20 at 03:26
  • 2
    @gvmani What kind of security issue exactly? My cursory experiments show that no code gets executed. Then, [here's](https://github.com/nodejs/node/blob/v15.4.0/lib/internal/modules/cjs/loader.js#L1143-L1151) the part that loads the json file. `JSONParse` is probably obtained by [copying the `JSON.parse` function](https://github.com/nodejs/node/blob/v15.4.0/lib/internal/per_context/primordials.js#L125-L129). So I'd say it's safe to assume that requiring json files is okay securitywise (nothing gets executed). – x-yuri Dec 18 '20 at 20:59
  • found this answer, and the file I wanted to try and load was called "test.json" nice work Travis you solved my exact request 2 years before I even knew I needed it – politus Oct 20 '21 at 13:49
62

Asynchronous is there for a reason! Throws stone at @mihai

Otherwise, here is the code he used with the asynchronous version:

// Declare variables
var fs = require('fs'),
    obj

// Read the file and send to the callback
fs.readFile('path/to/file', handleFile)

// Write the callback function
function handleFile(err, data) {
    if (err) throw err
    obj = JSON.parse(data)
    // You can now play with your datas
}
Florian Margaine
  • 58,730
  • 15
  • 91
  • 116
56

At least in Node v8.9.1, you can just do

var json_data = require('/path/to/local/file.json');

and access all the elements of the JSON object.

Alex Eftimiades
  • 2,527
  • 3
  • 24
  • 33
  • 10
    This approach loads file only once. If you will change the `file.json` after new require (without restarting program) data will be from first load. I do not have source to back this, but I had this in app I am building – Lukas Ignatavičius Mar 12 '18 at 18:22
  • Your answer is woefully incomplete. What that gets you is an object, and it doesn't even bother to implement tostring(). – David A. Gray Apr 09 '18 at 03:05
  • 5
    @DavidA.Gray The question wants to be able to access the objects as objects, not as strings. Asides from the singleton issue Lukas mentioned this answer is fine. – mikemaccana Jan 30 '19 at 11:20
  • 5
    Using require will also execute arbitrary code in the file. This method is insecure and I would recommend against it. – spoulson Apr 16 '19 at 12:46
54

Answer for 2022, using ES6 module syntax and async/await

In modern JavaScript, this can be done as a one-liner, without the need to install additional packages:

import { readFile } from 'fs/promises';

let data = JSON.parse(await readFile("filename.json", "utf8"));

Add a try/catch block to handle exceptions as needed.

Florian Ledermann
  • 3,187
  • 1
  • 29
  • 33
  • 1
    Where would you put the try catch? – F. Certainly. Aug 22 '21 at 22:08
  • 1
    I was looking for this, thank you! Works great when I know that the file's content is JSON data, but the extension is customized. The usual `require('./jsonfile.xyz')` cannot be used in this situation. – szegheo Dec 11 '21 at 13:48
  • 2
    Why not use `readFileSync` and remove the `await`? `JSON.parse(readFileSync("filename.json", "utf8"));` – Badal Saibo Mar 20 '23 at 07:14
20

In Node 8 you can use the built-in util.promisify() to asynchronously read a file like this

const {promisify} = require('util')
const fs = require('fs')
const readFileAsync = promisify(fs.readFile)

readFileAsync(`${__dirname}/my.json`, {encoding: 'utf8'})
  .then(contents => {
    const obj = JSON.parse(contents)
    console.log(obj)
  })
  .catch(error => {
    throw error
  })
jpsecher
  • 4,461
  • 2
  • 33
  • 42
  • 3
    `.readFile` is already async, if you're looking for the sync version, its name is `.readFileSync`. – Aternus May 26 '18 at 17:45
  • If you want to use promises, there's also `fs/promises` as of Node 10. Note: the API is experimental: https://nodejs.org/api/fs.html#fs_fs_promises_api – aboutaaron Sep 25 '18 at 18:25
  • @Aternus `.readFile` is *asynchronous*, but not `async`. Meaning, the function is not defined with `async` keyword, nor does it return a Promise, so you can't do `await fs.readFile('whatever.json');` – Kip Oct 01 '19 at 02:37
  • @Kip how about a CodeSandBox? – Aternus Oct 08 '19 at 14:39
8

Using fs-extra package is quite simple:

Sync:

const fs = require('fs-extra')

const packageObj = fs.readJsonSync('./package.json')
console.log(packageObj.version) 

Async:

const fs = require('fs-extra')

const packageObj = await fs.readJson('./package.json')
console.log(packageObj.version) 
Arturo Menchaca
  • 15,783
  • 1
  • 29
  • 53
7

using node-fs-extra (async await)

const readJsonFile = async () => {
    const myJsonObject = await fs.readJson('./my_json_file.json');
    console.log(myJsonObject);
}

readJsonFile() // prints your json object
ifelse.codes
  • 2,289
  • 23
  • 21
2

https://nodejs.org/dist/latest-v6.x/docs/api/fs.html#fs_fs_readfile_file_options_callback

var fs = require('fs');  

fs.readFile('/etc/passwd', (err, data) => {
  if (err) throw err;
  console.log(data);
});  

// options
fs.readFile('/etc/passwd', 'utf8', callback);

https://nodejs.org/dist/latest-v6.x/docs/api/fs.html#fs_fs_readfilesync_file_options

You can find all usage of Node.js at the File System docs!
hope this help for you!

xgqfrms
  • 10,077
  • 1
  • 69
  • 68
2
function parseIt(){
    return new Promise(function(res){
        try{
            var fs = require('fs');
            const dirPath = 'K:\\merge-xml-junit\\xml-results\\master.json';
            fs.readFile(dirPath,'utf8',function(err,data){
                if(err) throw err;
                res(data);
        })}
        catch(err){
            res(err);
        }
    });
}

async function test(){
    jsonData = await parseIt();
    var parsedJSON = JSON.parse(jsonData);
    var testSuite = parsedJSON['testsuites']['testsuite'];
    console.log(testSuite);
}

test();
Oliver Salzburg
  • 21,652
  • 20
  • 93
  • 138
2

Answer for 2022, using v8 Import assertions

import jsObject from "./test.json" assert { type: "json" };
console.log(jsObject)

Dynamic import

const jsObject = await import("./test.json", {assert: { type: "json" }});
console.log(jsObject);

Read more at: v8 Import assertions

Anil kumar
  • 1,216
  • 4
  • 12
0

So many answers, and no one ever made a benchmark to compare sync vs async vs require. I described the difference in use cases of reading json in memory via require, readFileSync and readFile here.

Jehy
  • 4,729
  • 1
  • 38
  • 55
-1

If you are looking for a complete solution for Async loading a JSON file from Relative Path with Error Handling

  // Global variables
  // Request path module for relative path
    const path = require('path')
  // Request File System Module
   var fs = require('fs');


// GET request for the /list_user page.
router.get('/listUsers', function (req, res) {
   console.log("Got a GET request for list of users");

     // Create a relative path URL
    let reqPath = path.join(__dirname, '../mock/users.json');

    //Read JSON from relative path of this file
    fs.readFile(reqPath , 'utf8', function (err, data) {
        //Handle Error
       if(!err) {
         //Handle Success
          console.log("Success"+data);
         // Parse Data to JSON OR
          var jsonObj = JSON.parse(data)
         //Send back as Response
          res.end( data );
        }else {
           //Handle Error
           res.end("Error: "+err )
        }
   });
})

Directory Structure:

enter image description here

Hitesh Sahu
  • 41,955
  • 17
  • 205
  • 154