Count The Number Of Json Key Pairs In A Bash Shell Script

Question

Within a ADO yaml pipeline one of my jobs has a inline script bash shell running. I want to automate a counter being used for a loop later in the script. The counter represents the number of some json embedded key pairs (the value of these key pairs hold variables). Like in most ADO pipelines the variable sheet (it's actually a variable template sheet but that doesn't matter) is stored separately to the pipeline and called on at the start of the yaml pipeline.

{
  "abc": {
    "models": {
      "model1": {
        "a": "x",
        "b": "z"
      },
      "model2": {
        "a": "x",
        "b": "z"
      },
      "model3": {
        "a": "x",
        "b": "z"
      }
    }
  }
}

The desired outcome of this example would be 3 but in the future more models will be added. For context the model names are subject to change - so it's not possible to do something odd like take the last key pair name and split off the number. It would be best if the solution is written in Bash as I would prefer to avoid any complexities.

The only related things I could find and test were:

            modelCount5="$(jq 'abc.models | length' ${{ parameters.variableGroup }})"
            echo $modelCount5
            modelCount6="$(jq '.abc.models | length' ${{ parameters.variableGroup }})"
            echo $modelCount6
            modelCount7="$(jq '$(abc.models) | length' )"
            echo $modelCount7
            modelCount9="$(jq '$(abc.models) | length' ${{ parameters.variableGroup }})"
            echo $modelCount9

They all resulted in complie errors or directory not found errors. Eg:

jq: 1 compile error
or
jq: error: Could not open file CTS: No such file or directory

I have checked the agent running the pipeline and jq has been preinstalled.

Answer

Thankyou all for your help. In the end I did this:

   modelCount="$(jq '.variables.abc.models | length' $(Build.SourcesDirectory)/variables/templates/variables-sheet.jsonc)"

...and it worked great! It was a matter of configuring my path to the file properly.

To explain _why_ you need to make the change pmf suggests: You aren't actually running `jq` at all in the first place right now. `modelCount=jq ...` doesn't start jq; instead, it temporarily sets a variable named `modelCount` that has `jq` as its value for the duration of the larger `...` command. As such, the problem you're hitting isn't json- or jq- or azure-specific; it's a general bash syntax problem. — Charles Duffy, May 10 '22 at 16:00
@CharlesDuffy Okay but I'm trying to save the result in a variable to be used later — James Lahiffe Baker, May 10 '22 at 16:03
FYI, you are missing a closing `}`, and you shoudl remove the `,` after each `"b": "z"` line. — Nic3500, May 10 '22 at 16:07
@Nic3500 thankyou, this is just some dummy text instead of using the real thing as it adds unneeded complexity — James Lahiffe Baker, May 10 '22 at 16:12
@JamesLahiffeBaker, yes, and pmf's comment tells you how to do that, as does the duplicate I linked describing the general-case correct practice. — Charles Duffy, May 10 '22 at 16:12
@CharlesDuffy thanks for your help but I'm getting a 'jq: 1 compile error' error now. — James Lahiffe Baker, May 10 '22 at 17:00
Hey. I'm the "moderator" (tag-specific gold-badge, vs the true unrestricted moderators who have diamonds) who tagged this as a duplicate, and as of yet, you haven't edited the question to show that you've applied the fix given in the duplicate (which is to make it `modelCount=$(jq ...)`, as pmf instructed you). Only **after you apply that fix** and [edit] to show the new problem you still have will the question no longer be a duplicate; when that's done I'll gladly reopen it (if there isn't a new, more-applicable duplicate for that edited question). — Charles Duffy, May 10 '22 at 17:11
Well I didn't really get a chance to do so. I have to run a pipeline and test your suggestion? Seems crazy to shut it down to anymore ideas because of some syntax issues. In the end your suggestion hasn't worked, I'm sorry too say. If you were to have left the question open maybe others could have chipped in with a different suggestion. the duplicate wasn't remotely similar to the main vibe of the question. — James Lahiffe Baker, May 10 '22 at 17:34
The modelCount6 example works fine when I run it. See https://replit.com/@CharlesDuffy2/DevotedDarkvioletMaps#main.sh — Charles Duffy, May 10 '22 at 18:11
...which does perhaps mean it's a azure pipeline problem (on that one, after you remove the other more-broken attempts; 7 and 9 will never work). — Charles Duffy, May 10 '22 at 18:12
What I'd suggest doing here, to help you build a question that can be followed by people who don't know how your Azure setup is configured, is putting `set -x` earlier in your script to enable trace-level logging, so someone reading those logs can see if `${{ parameters.variableGroup }}` is spitting out a filename, or a bunch of JSON, or something else / etc. — Charles Duffy, May 10 '22 at 18:13
To be clear about why this was closed in its prior form: Each Stack Overflow question should be about _only one technical problem_. If that immediate problem is already in the knowledgebase, the question is a duplicate. If you have a different, less-immediate problem, then you should be asking about _that_ instead, but part of the work of writing a question is demonstrating a single, specific, narrowly-defined technical issue. We're a Q&A database, not a help forum, and a good FAQ has questions that are as narrow and well-defined as possible. — Charles Duffy, May 10 '22 at 18:18
Thankyou all for your help. In the end I did this ``` modelCount="$(jq '.variables.abc.models | length' $(Build.SourcesDirectory)/variables/templates/variables-sheet.jsonc)" ``` and it worked great! @CharlesDuffy — James Lahiffe Baker, May 12 '22 at 11:47

score 0 · Answer 1 · answered Aug 14 '22 at 09:16

You can get the extract the count and store it in a shell variable with the following statement:

modelCount="$(jq '.variables.abc.models | length' path/to/your/file"

length in jq gives you the length of an array. $(…) is command substution and substitutes the command with its output to be used by the parent command line.

Count The Number Of Json Key Pairs In A Bash Shell Script

1 Answers1