132

We're building a website using the Pentaho CTools library, which has a graphical dashboard editor which writes out JSON-format files for part of the dashboard.

I'd like to apply a transform to these files before check-in to git in order to sort them by key and then by the value of certain keys. The purpose is to make diffs easier, since the editor has a habit of rearranging all of the json fields.

For example, we might have something like this:

{
  "components": {
    "rows": [
      {
        "id": "CHARTS",
        "name": "Charts",
        "parent": "UnIqEiD",
        "properties": [
          {
            "name": "Group",
            "type": "Label",
            "value": "Charts"
          }
        ],
        "type": "Label",
        "typeDesc": "<i>Group</i>"
      },
      {
        "id": "kjalajsdjf",
        "meta_cdwSupport": "true",
        "parent": "CHARTS",
        "properties": [
          {
            "name": "name",
            "type": "Id",
            "value": "Value1"
          },
          {
            "name": "title",
            "type": "String",
            "value": "Value2"
          },
          {
            "name": "listeners",
            "type": "Listeners",
            "value": "[]"
          },
...

We are able to jq --sort-keys (http://stedolan.github.io/jq/) to sort all of the keys, but I'm struggling to find out how to use the sort_by function to then sort certain specific elements by the value of certain keys (so, in the example above, sorting by properties.name for example. Any ideas?

memoselyk
  • 3,993
  • 1
  • 17
  • 28
karlos
  • 3,965
  • 3
  • 16
  • 19

3 Answers3

136

Ok with some assistance on the IRC channel I've found an answer.

Basically, it looks like this:

jq \
  '.components.rows|=sort_by(.id)|.components.rows[].properties|=sort_by(.name)' \
  file.json > out.json

Select the right object,
walk into arrays if needed,
then sort_by a single value.

I was trying sort_by(.components.rows.id) which failed.

|= instead of | passes the values along instead of stripping them.

Rob Bednark
  • 25,981
  • 23
  • 80
  • 125
karlos
  • 3,965
  • 3
  • 16
  • 19
  • Note: for anyone who actually wants to do this with a CTools `.cdfde` file, sorting the keys seems to be ok, as does sorting the `name` elements inside the properties array, but sorting the `component` `rows` by `id` is a no-no, it breaks stuff. So don't do it... – karlos May 22 '15 at 21:29
  • does not work on arbitrary json. The solution from @thibaud-ledent does. – 2xMax Mar 10 '22 at 22:50
  • 3
    @2xMax It's not designed to work on 'arbitrary' JSON - it needs to be modified to sort on the specific keys you need. – Tim Malone Jul 04 '22 at 01:20
  • That `|=` thing strikes me as the key here. Thanks for this! Glad you found an answer, and now I have one, too. I'll add that while not relevant to your particular example data, in some cases it might also be useful to do `sort_by(.some_query_identifier|tonumber)` -- to sort numerically by whatever field (`.id` as the `.some_query_identifier` in your case, but those weren't strictly numeric). – lindes Jan 02 '23 at 20:55
  • I also found -s "slurp" helpful, see https://stackoverflow.com/a/59354557/34935 – dfrankow Mar 21 '23 at 14:55
114

This doesn't answer the question, but here is another way to sort by attributes/keys:

jq --sort-keys . my_file > sorted_file

-or-

jq -S . my_file > sorted_file
Rob Bednark
  • 25,981
  • 23
  • 80
  • 125
Thibaud Ledent
  • 1,446
  • 1
  • 7
  • 6
  • 11
    This is the same as `--sort-keys` which was mentioned in the question. – Timmmm May 11 '21 at 13:43
  • 1
    But you see it has so many upvotes because most of the people here are here because we just want the root level keys sorted :) – Steven Lu Dec 14 '22 at 02:20
0

If you need it inside a JQ query go for:

def sort_keys:
    (if (.|type!="object") then error("sort_keys: object expected") else . end)
    |(.|keys) as $keys
    |. as $o
    |reduce $keys[] as $i ({};.[$i]=$o[$i]);
Black Rain
  • 61
  • 1
  • 2