Comparing two json files : shell scripting

Question

I would like to compare two json files which look like the following:

[
   {
      "type" : 1,
      "children" : {
         "nsubj" : {
            "role" : "topic",
            "POS" : [
               "noun"
            ]
         }
      },
      "role" : "vehicle",
      "POS" : [
         "noun"
      ]
   },

and the other is in the similar format, but there are some differences between the two because one json file is made up of 3336 lines, while another is made up of 3724 lines. I would like to write a shell script which would compare the two line by line and whenever it finds a difference, output the line number where the difference occurred.

score 26 · Answer 1 · answered May 08 '17 at 22:42

26

To compare json files you should convert them so they have same order of keys. Very good tool for this job is jq (https://stedolan.github.io/jq/) where you can do:

jq -S . fileA.json > fileA_fmt.json
jq -S . fileB.json > fileB_fmt.json

then, you can use your favourite tool for text file comparison. I like kdiff3 for GUI or just plain diff when in pure command-line e.g.:

diff fileA_fmt.json fileB_fmt.json

answered May 08 '17 at 22:42

bartolomeon_n

373
3
6

2

Very good tool indeed. Just `brew install jq` to install it on MacOS – phadjido Jul 05 '19 at 13:21
1

One thing to note is that you have to use a different file for the output. I attempted to write over the same file and it ends up leaving you with an empty file. – Qwertie Oct 28 '19 at 03:15

score 16 · Accepted Answer · answered Dec 31 '13 at 01:24

16

Just use diff. Like in

diff --unified file1.json file2.json

answered Dec 31 '13 at 01:24

Sergii Dymchenko

6,890
1
21
46

Yes, I already tried this, but I am not too sure what the meaning of the headings are like @@ -1,2841 +1,2721 @@ for example – gkumar7 Dec 31 '13 at 01:42
If you don't use 'unified' flag, then you'll see something like "9c9". It means line 9 changed. – Sergii Dymchenko Dec 31 '13 at 02:01
could you explain the notation for the unified flag, basically is does it mean lines 1 to 2841 need to be changed? and similarly lines 1 to 2721 need to be changed in the second json file? – gkumar7 Dec 31 '13 at 02:04

score 7 · Answer 3 · edited May 11 '21 at 17:42

7

Just to update on the answer from bartolomeon_n, you can actually do this all on one line.

diff <(jq -S . fileA.json) <(jq -S . fileB.json)
# or, with nice columns and colours:
diff -y --left-column --color <(jq -S . fileA.json) <(jq -S . fileB.json)

edited May 11 '21 at 17:42

andrew lorien

2,310
1
24
30

answered Nov 22 '20 at 01:19

csaroff

83
2
9

1

Great! Also removes the need to create (and clean up) of temporary files. – FruitBreak Mar 09 '21 at 09:45

Comparing two json files : shell scripting

3 Answers3

Linked