Finding a substring from a JSON attribute with jq

Question

I know how to retrieve an entire JSON attribute with jq, but I want to extract only a specific substring. Consider the following sample input:

[
  {
    "name": "test",
    "output": "",
    "error": "",
    "state": "unknown",
    "startTime": 1571292623936,
    "endTime": 0,
    "extra": {},
    "warning": "************************* test Warnings *************************\n*\n* \n*****************************************************************",
    "hasWarning": false
  },
  {
    "name": "npm run test",
    "output": "\n> DISPLAY was set to: \":99\"\n\nCypress will attempt to fix the problem and rerun.\n\n\n            Running: consumer/oct.js...                                          (1 of 1) \nPROCESSING JS RESOURCE FILE   FROM:/PMT1469/workspace/E2EI/cypress/e2e/consumer/kindle.js\n{\"dataFile\":\"scripts/regression/transfers/card/kindle.csv\"}\nSENDING JS RESOURCE FILE FROM: /PMT-1469/workspace/E2E-UI { startedTestsAt: '2019-10-17T06:10:59.339Z',\n  endedTestsAt: '2019-10-17T06:11:53.542Z',\n totalDuration: 54203,\n  totalSuites: 4,\n  totalTests: 2,\n  totalFailed: 2,\n  totalPassed: 0,\n  totalPending: 0,\n  totalSkipped: 0,\n\n  browserPath: '',\n  browserName: 'electron',\n reporter: 'mochawesome',\n         taskTimeout: 60000,\n     video: true,\n      known: true }\n",
    "error": null,
    "state": "success",
    "startTime": 1571292631223,
    "endTime": 1571292718780,
    "extra": {},
    "warning": "************************* npm run test Warnings *************************\n*\n* \n*************************************************************************",
    "hasWarning": false
  }
]

I just want to pick the following values in the above JSON payload which is in "output" attribute.

Expected output:

totalDuration: 54203
totalSuites: 4
totalFailed: 2
totalPassed: 0
totalSkipped: 0

We can easily fetch the attribute values using jq -r '.[].output', but I'm trying to only capture substrings of the form total<something>: <number>.

...whereas if you're asking for something that is more interesting or distinguishable, please [edit] the question to make that clear. (I've closed it for the moment, but if made clearly distinct, it's easy enough to reopen, or to modify the duplicate list to have something more on-point). — Charles Duffy, Oct 17 '19 at 17:55
I just edited the question. I knew how to parse json and fetch the value by using jq library. But this is little different. There is an attribute called "Output" where it contains my expected results along some more text. I just want to filter my expected result and get them as result. Hope this makes sense. — ArrchanaMohan, Oct 17 '19 at 17:58
@oguzismail, you'll notice that the second half of my answer does use `scan`. :) — Charles Duffy, Oct 17 '19 at 22:49

score 3 · Accepted Answer · answered Oct 17 '19 at 19:18

The inefficient-but-easy answer is to do the bulk of the work in a separate pipeline stage. Assuming GNU tools:

jq -r '.[].output' <in.json \
  | grep -Eo '^[[:space:]]+(total[[:alpha:]]+: [[:digit:]]+)' \
  | sed -re 's/^[[:space:]]+//'

However, with modern jq, one can do much better:

jq -r '.[].output | scan("total[[:alpha:]]+: [[:digit:]]+")' <in.json

Finding a substring from a JSON attribute with jq

1 Answers1