Convert text file into clean JSON format

Question

I want to read a text file and write a text file in a clean JSON format.

My text file looks like this

Version:2
TaxiStation:6072
TaxiLicense:EM9543
System_ID:910

When I use the following:

data2= JSON.stringify(data2, null, position+1);
fs.writeFile(dirarray[0], data2, function(err) { ...

My output always comes out ugly : "Version:2\r\nTaxiStation:6072\r\nTaxiLicense:EM8378\r\n

Ideally, I would want to make my text file formatted nicely. Is it possible to achieve this?

{
   "Version":2,
   "TaxiStation":6072,
   "TaxiLicense":"EM9543",
   "System_ID":910
}

Have you tried my answer? To give you can in-context example, I've tried adding it to your code posted in another comment: http://pastebin.com/5KWM7kmB Does this work for you? — Mikuso, May 09 '14 at 10:33

score 3 · Answer 1 · answered May 09 '14 at 10:03

3

Use something like this to transform your data before you stringify it:

data2 = data2.split(/\r?\n/).reduce(function(m,i){
    var s = i.split(':');
    m[s.shift()] = s.join(':');
    return m;
}, {});

JSON.stringify(data2);

answered May 09 '14 at 10:03

Mikuso

1,330
2
11
17

nice one :) you could also do `m[s[0]] = parseInt(s[1]) || s[1];` to add support for parsing string/number – Jos de Jong May 09 '14 at 11:42
Good thought, but this would round numbers like `1.234` to `1`, and it would interpret strings like `"56FGHW99"` as `56`. Unfortunately, unlike JSON, the format of his logs do not indicate a data type. So, best to keep it as a string to be safe. – Mikuso May 09 '14 at 11:59
It would be worthwhile adding checks for empty lines or lines which do not conform to `KEY:VALUE` pairs. – Mikuso May 09 '14 at 11:59
Nope. Some strings will still be interpreted as numbers, even when they're supposed to be strings. If you have a string like "5E2", you'll then have a number in your output: 500. This is a corruption / loss of data. See my comments on @Enrichman 's answer for more examples of why this fails. – Mikuso May 09 '14 at 12:39
Sorry for the late reply, I'll test this out shortly. – krikara May 12 '14 at 02:36
Upvoted. This is near perfect. Is there a way to retain number values for example `"Version":2` as opposed to `"Version":"2"` without encountering that problem above where you mentioned how `56FGHW99` would be converted into `56` ? – krikara May 12 '14 at 09:56
There isn't really a safe way to do it without possibly encountering other problems later on. It's best in this case to typecast based on the property name. i.e. `if (key === "Version") value = parseInt(value);`. Guessing types is a bad idea. – Mikuso May 13 '14 at 07:49

Enrichman · Answer 2 · 2014-05-09T11:19:28.707

-1

The main problem is that the stringify function will take a js object and jsonize it, but your file is not an object, or array, or anything, so, once read, it's interpreted like a single string. And so it's parsed as a string!

From your file you should create a js object matching your lines and then parse it.

This is a working example: http://pastebin.com/EhnhNUyB

The main part is here

dataArr = data2.split('\n');

var obj = {};

for (i=0;i<dataArr.length;i++) {
    keyVal = dataArr[i].split(":");

    var key = keyVal[0];

    var val;
    if(isInt(keyVal[1])) {
        val = parseInt(keyVal[1]);
    } else {
        val = keyVal[1];
    }
    obj[key] = val;
}

data2 = JSON.stringify(obj, null, 4);

as you can see I've created a new object and pushed to it the new key-values.

edited May 09 '14 at 11:19

answered May 09 '14 at 07:59

Enrichman

11,157
11
67
101

Can you show what your file looks like? Notepad and Wordpad are showing the same results for me. Also, `json.stringify` doesn't seem to handle all my strings within the file, even when I add in all the commons to the original text file. – krikara May 09 '14 at 08:12
Actually seems that you're doing it right: [http://stackoverflow.com/questions/5670752/write-pretty-json-to-file-using-node-js]. Can you upload the file so I can check if for me is ok? – Enrichman May 09 '14 at 08:33
There are two main differences with that link. The first is that the OP reads a JSON file where my file is just regular text. The second is that I actually need write to a file whereas the correct marked answer simply outputs. – krikara May 09 '14 at 08:39
With that said, here is my code. The text file is the exact same as in my post. http://pastebin.com/iTg8G6Lp – krikara May 09 '14 at 08:39
@krikara Edited. I'm sorry to not provide a "full code" answer but I'm a bit busy now. I hope to answer you better soon. :) – Enrichman May 09 '14 at 09:07
1

You haven't included the isInt() function in your answer, but I don't think it's wise to guess the data type of the value in the pair anyway. Keep each value as a string, or else you could end up guessing wrongly and "corrupting" data. – Mikuso May 09 '14 at 12:03
1

For example, a string such as "1E1" would pass your test of `isInt()`, and when passed to `parseInt()` will return 1. – Mikuso May 09 '14 at 12:13
1

Second example, a UK telephone number such as `"01234567890"` would pass the `isInt()` test, and return 1234567890 after been fed through `parseInt()` (losing the 0 at the beginning of the string) – Mikuso May 09 '14 at 12:14
1

Third example. Due to precision issues with javascript and big numbers, isInt() would return true for "9999999999999999999", but parseInt() would return 10000000000000000000 instead. This, again, is a loss/corruption of data. – Mikuso May 09 '14 at 12:16
@Mikuso It was included in the pastebin link, I found it on StackOverflow so I'm not sure but I guess it will be correct most of the times. Anyway, it's a working example for the data provided, of course if you want a "real" JSON parsing you should do lot of different check for different types! Thanks for the notes. :) – Enrichman May 09 '14 at 12:57
2

@Enrichman The beauty of JSON is that the type doesn't have to be guessed (as the type can be inferred by context) and the datatypes supported by JSON are a subset of those supported by Javascript. However, you can't count on either of these with data like this. This exercise of "guess the datatype and typecast" is not the best approach here. He should only typecast values when he knows that the corresponding value should contain a certain type. – Mikuso May 09 '14 at 13:15
1

@Enrichman One final example in case I haven't yet convinced you. Say he has an input such as `TaxiLicense:EM9543` and parses it into the form `{"TaxiLicense":"EM9543"}`. Let's assign this object to a variable called `data2`. He may then perform some other action on the data which he would expect to succeed: `data2.TaxiLicense.split("M") // ["E", "9543"]`. Now, say he encountered another string in his input file `TaxiLicense:19543`. If the parser believes this should be an integer rather than a string, then his later call to `.split()` will fail, as `Number.prototype.split` does not exist. – Mikuso May 09 '14 at 13:28
1

@Mikuso no, I've understand the "problem", the problem is that from the data extracted like this you cannot create directly a JSON because it doesn't make really sense. Btw, since I guess the op has a good knowledge about the datas will do the right choice and decide to, or not to, cast. – Enrichman May 09 '14 at 13:32

Convert text file into clean JSON format

2 Answers2