How can I reformat this simple JSON so it doesn't catch "Circular structure to JSON" exception?

Question

Introduction

I'm learning JavaScript on my own and JSON its something along the path. I'm working on a JavaScript WebScraper and I want, for now, load my results in JSON format.

I know I can use data base, server-client stuff, etc to work with data. But I want to take this approach as learning JSON and how to parse/create/format it's my main goal for today.

Explaining variables

As you may have guessed the data stored in the fore mentioned variables comes from an html file. So an example of the content in:

users[] -> "<a href=\"countries.php?id=1\">Egypt</a>"
GDP[] -> "<td> $2,971</td>"
Regions[] -> "<td> Egypt </td>"
Align[] -> "<td> Eastern Bloc </td>"

Code

let countries = [];
for(let i = 0; i < users.length; i++)
{
    countries.push( {
        'country' : [{
            'name' : users[i],
            'GDP' : GDP[i],
            'Region' : regions[i],
            'Align' : align[i]

    }]})
};



let obj_data = JSON.stringify(countries, null, 2);
fs.writeFileSync('countryballs.json', obj_data);

Code explanation

I have previously loaded into arrays (users, GDP, regionsm align) those store the data (String format) I had extracted from a website.

My idea was to then "dump" it into an object with which the stringify() function format would format it into JSON.

I have tested it without the loop (static data just for testing) and it works.

Type of error

let obj_data = JSON.stringify(countries, null, 2);
                        ^

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Node'
    |     property 'children' -> object with constructor 'Array'
    |     index 0 -> object with constructor 'Node'
    --- property 'parent' closes the circle

What I want from this question

I want to know what makes this JSON format "Circular" and how to make this code work for my goals.

Notes

• I am working with Node.js and Visual Studio Code

EDIT

This is further explanation for those who were interested and thought it was not a good question.

Test code that works

let countries;
    console.log(users.length)
    for(let i = 0; i < users.length; i++)
    {
        countries = {
            
            country : [
                {
                    "name" : 'CountryTest'
                }
            ]
        } 
            
    };
     
    let obj_data = JSON.stringify(countries, null, 2);
    fs.writeFileSync('countryballs.json', obj_data);    
});

Notice in comparison to the previous code, right now I am inputing "manually" the name of the country object.

This way absolutely works as you can see below:

Now, if I change 'CountryTest' to into a users[i] where I store country names (Forget about why countries are tagged users, it is out of the scope of this question)

It shows me the previous circular error.

A "Partial Solution" for this was to add +"" which, as I said, partially solved the problem as now there is not "Circular Error"

Example:

 for(let i = 0; i < users.length; i++)
    {
        countries = {
            
            country : [
                {
                    "name" : users[i]+''
                }
            ]
        }   
    };

Resulting in:

Another bug, which I do not know why is that only shows 1 country when there are 32 in the array users[]

This makes me think that the answers provided are not correct so far.

Desired JSON format

{
  "countries": {
    "country": [
      {
        "name": "",
        "GDP" : "",
        "Region" : "",
        "Align" : ""
      },
      {
        "name": "",
        "GDP" : "",
        "Region" : "",
        "Align" : ""
      },
      {
        "name": "",
        "GDP" : "",
        "Region" : "",
        "Align" : ""
      }
      
    ]}
}

Answer 1

Circular structure error occurs when you have a property of the object which is the object itself directly (a -> a) or indirectly (a -> b -> a).

To avoid the error message, tell JSON.stringify what to do when it encounters a circular reference. For example, if you have a person pointing to another person ("parent"), which may (or may not) point to the original person, do the following:

JSON.stringify( that.person, function( key, value) {
  if( key == 'parent') { return value.id;}
  else {return value;}
})

The second parameter to stringify is a filter function. Here it simply converts the referred object to its ID, but you are free to do whatever you like to break the circular reference.

You can test the above code with the following:

function Person( params) {
  this.id = params['id'];
  this.name = params['name']; 
  this.father = null;
  this.fingers = [];
  // etc.
}

var me = new Person({ id: 1, name: 'Luke'});
var him = new Person( { id:2, name: 'Darth Vader'});
me.father = him; 
JSON.stringify(me); // so far so good

him.father = me; // time travel assumed :-)
JSON.stringify(me); // "TypeError: Converting circular structure to JSON"
// But this should do the job:
JSON.stringify(me, function( key, value) {
  if(key == 'father') { 
    return value.id;
  } else {
    return value;
  };
})

The answer is from StackOverflow question,

Stringify (convert to JSON) a JavaScript object with circular reference

Answer 2

From your output, it looks as though users is a list of DOM nodes. Rather than referring to these directly (where there are all sort of possible cyclical structures), if you just want their text, instead of using users directly, try something like

country : [
    {
        "name" : users[i].textContent // maybe also followed by `.trim()
    }
]

Or you could do this up front to your whole list:

const usersText = [...users].map(node => node.textContent)

and then use usersText in place of users as you build your object.

If GDP, regions and align are also references to your HTML, then you might have to do the same with them.

Answer 3

EUREKA!

As some of you have mentioned above, let me tell you it is not a problem of circularity, at first..., in the JSON design. It is an error of the data itself.

When I scraped the data it came in html format i.e <td>whatever</td>, I did not care about that as I could simply take it away later. I was way too focused in having the JSON well formatted and learning.

As @VLAZ and @Scott Sauyezt mentioned above, it could be that some of the data, if it is not well formatted into string, it might be referring to itself somehow as so I started to work on that.

Lets have a look at this assumption...

To extract the data I used the cheerio.js which gives you a kind of jquery thing to parse html.

To extract the name of the country I used:

nullTest = ($('table').eq(2).find('tr').eq(i).find('td').find('a').last());
            //"Partial solution" for the OutOfIndex nulls
            if (nullTest != null)
            {
                users.push(nullTest);                
            }

(nullTest helps me avoid nulls, I will implement some RegEx when everything works to polish the code a bit)

This "query" would output me something like:

<a href="source">whatEverIsInHereIfThereIsAny</a>

or else.

to get rid off this html thing just add .html() at the end of the "jquery" such as:

($('table').eq(2).find('tr').eq(i).find('td').find('a').last().html());

That way you are now working with String and avoiding any error and thus solves this question.