Transfer or merge only some properties from one JSON file to another with jq

Question

I have two JSON files:

$ jq . a.json b.json 
{
  "id": "ZGVhZGJlZWY=",
  "name": "first file",
  "version": 1,
  "description": "just a simple json file"
}
{
  "version": 2,
  "name": "fake name",
  "dependencies": [
    4,
    2
  ],
  "comment": "I'm just sitting here, ignore me"
}

and want to merge them into a single file (think of file 1 as "template" and file 2 as "actual values"). I don't want to merge all properties, I only want to transfer some properties of the second file (specifically only version and dependencies). version should overwrite the value in the original file and dependencies should be added to the new file. name must not be overwritten and the original name must be kept.

This is the expected result:

{
  "id": "ZGVhZGJlZWY=",
  "name": "first file",
  "version": 2,
  "description": "just a simple json file",
  "dependencies": [
    4,
    2
  ]
}

I know that jq supports the + and * operators to merge or merge recursively, but how can I apply those to only some properties and not all? How can I access both files in my jq program; do I have to preprocess the file and then use --arg in a second jq call?

Obviously, jq '. + {version, dependencies}' a.json b.json does not work. What is the correct program here?

What would the solution look like if description should also be dropped from the output?

peak · Answer 1 · 2023-02-08T22:16:13.690

3

If you want simplicity, brevity, and efficiency, consider:

jq '. + (input|{version, dependencies})' a.json b.json

If the first file might have .dependencies, and if in that case you want to add in the second file's:

jq '. as $a | input as $b | $a + ($b|{version}) | .dependencies += $b.dependencies'

To drop .description, you could append | del(.description) to either of these filters.

edited Feb 08 '23 at 22:16

answered Feb 08 '23 at 22:09

peak

105,803
17
152
177

Thanks, I always forget about the very useful `input` filter. "One input == one file", is that correct? Or is there special care required if the file contains multiple entities? – knittl Feb 09 '23 at 06:59
One `input` reads one JSON entity. It is the singular of `inputs` :-) – peak Feb 09 '23 at 07:38
I tried with `b.json` containing two JSON entities (i.e. a stream of two JSON objects). This will abort with an error (but still output a result). Using a variable would always use the first entity (with `.[0]`) or build the product with `.[]`. – knittl Feb 09 '23 at 07:49
If b.json has two JSON entities, then after the first of these has been read, `.` will read the second, and then `input` will fail. What did you expect? – peak Feb 09 '23 at 08:47
I'd expect `input` to return the first entity of the file; looks like my expectation is wrong/unrealastic. But `.` will always read the first file, `input` the file after, no? I guess my confusion comes from "does `input` read a file or an entity and how does it read a file with multiple entities?" – knittl Feb 09 '23 at 08:55
Are you expecting `input` to backtrack??? It doesn't. `input, input` would read two entities if there are two. You might like to look at how `inputs` is implemented. By the way, using `first(inputs)` is sometimes preferable to using `input` as the latter gives an error if there is nothing to read. – peak Feb 09 '23 at 09:33
But there is something to read?? I'm simply not reading all. I have 3 entities (1 entity in file a, 2 entities in file b). Now, with program `[., input]` I get an error (»jq: error (at ...): break«), but I'd expect none: there are enough entities to read. `[., input, input]` works. `[., input, input, input]` doesn't either, but that's what I would expect. – knittl Feb 09 '23 at 09:40
To get a better understanding, try using debug, e.g. `[debug, "input: \(input)"]` – peak Feb 09 '23 at 09:48

score 1 · Answer 2 · answered Feb 08 '23 at 21:33

+ or * can be used here, correct. Let's first see how + works:

$ jq -n '{a:1,b:2} + {b:3,c:4}'
{
  "a": 1,
  "b": 3,
  "c": 4
}

Properties only present in the left object are kept
Properties of the right object overwrite properties of the left object
Properties only present in the right object are added

Perfect, now how to get the objects from two unrelated files? --slurpfile can be used, which reads all JSON entities in the file into an array and puts it into a variable.

$ jq --slurpfile b b.json '. + $b[0]' a.json
{
  "id": "ZGVhZGJlZWY=",
  "name": "fake name",
  "version": 2,
  "description": "just a simple json file",
  "dependencies": [
    4,
    2
  ],
  "comment": "I'm just sitting here, ignore me"
}

We are getting closer, but are not quite there yet. name is overwritten and comment is added; both of which we do not want. To solve this, we can transform the slurped object into a new object which only contains the properties we care about

$ jq --slurpfile b b.json '. + ($b[0] | {version,dependencies})' a.json
{
  "id": "ZGVhZGJlZWY=",
  "name": "first file",
  "version": 2,
  "description": "just a simple json file",
  "dependencies": [
    4,
    2
  ]
}

Now let's address part two of the question: "can some properties of the first file be dropped?"

There are basically two options:

Creating a new object containing only the required properties and then adding the second object (any property part of the second file can be ignored, since it will be added anyway): {id,name} + ($b[0] | {version,dependencies})
Deleting the unneeded properties: del(.description) + ($b[0] | {version,dependencies}) or . + ($b[0] | {version,dependencies}) | del(.description)

Depending on the number of properties you want to keep/drop, one or the other solution might be simpler to use. Creating a new object has the advantage of being able to rename properties in one go.

Executing solution 2:

$ jq --slurpfile b b.json 'del(.description) + ($b[0] | {version,dependencies})' a.json
{
  "id": "ZGVhZGJlZWY=",
  "name": "first file",
  "version": 2,
  "dependencies": [
    4,
    2
  ]
}

Transfer or merge only some properties from one JSON file to another with jq

2 Answers2

Linked

Related