Github Action - After using jq to extract a json value the following bash commands still attempt to execute inside jq

Question

So I have a bash command like

- name: Get waiting pull_request_id
        id: get_pr_id
        run: |
          prId='${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq '.workflow_runs' | jq '.[] | first(select(.status=="in_progress"))' | jq '.pull_requests' | jq '. | first' | jq '.number' \
          echo "$prId" \
          echo "prId=$prId" >> $GITHUB_OUTPUT

I am using this to lookup a workflow that is in progress and extract the pull request Id. I want to output the pull requestId to use in the next step

The error is

jq: error: Could not open file echo: No such file or directory
jq: error: Could not open file : No such file or directory
jq: error: Could not open file echo: No such file or directory
jq: error: Could not open file prId=: No such file or directory

So it seems that even with the \ I am still in the context of the previous line of script. I have tried semicolons to no avail. How can I make this work?

If I remove the backslashes, then the first line never seems to query anything or even execute as I get no output at all

I have verified that the first cmd line does return the prId if I echo it instead of assigning to a variable. I have tried to echo into the github output directly from that 1st line, but it requires quotation marks and I can't figure out how to do a jq select without using quotes.

On windows or linux? Could be that this is running PowerShell, not bash. https://stackoverflow.com/questions/74443940/value-not-set-using-github-output/74444094#74444094 — jessehouwing, Dec 12 '22 at 21:23
It's linux, I know it's bash because jq works if I echo the value out directly from that 1st cmd — Josh, Dec 12 '22 at 21:30

knittl · Accepted Answer · 2022-12-12T22:17:00.903

1

prId='${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq ...

Does not provide any input to jq. It sets a variable, then pipes nothing to jq. Setting a variable does not produce any output.

If you want to feed the variable to jq, you have to output it explicitly:

prId='${{ steps.get_in_progress_workflow_run.outputs.data }}';
printf '%s\n' "$prId" | jq ...

If that does not work, you could try the following:

prId='${{ steps.get_in_progress_workflow_run.outputs.data }}';
echo '${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq ...

... | jq '.number' \
          echo "$prId" \
          echo "prId=$prId" >> $GITHUB_OUTPUT

Backslashes at the end of a line escape the newline character and continues the line to the next line. So the 3 lines above are parsed as a single line by your shell interpreter.

Also note that there's no need to start that many separate jq processes. Your shell pipeline can easily be merged into a single jq invocation:

... | jq '.workflow_runs[] | first(select(.status=="in_progress")) | .pull_requests[0] | .number'

After several comments and trying to understand the problem from a different angle, here's what are probably really trying to do:

prId="$(echo '${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq '.workflow_runs[] | first(select(.status=="in_progress")) | .pull_requests[0] | .number')";
echo "$prId";
echo "prId=$prId" >> "$GITHUB_OUTPUT"

The magic word you are looking for here is command substitution

edited Dec 12 '22 at 22:17

answered Dec 12 '22 at 21:41

knittl

246,190
53
318
364

If I run it exactly as you wrote it I get "/usr/bin/jq: Argument list too long Error: writing output failed: Broken pipe" – Josh Dec 12 '22 at 21:57
If I change the printf '%s' "$prId" to printf '%s' '$prId' (single quotes around prId) then I get a different error "parse error: Invalid numeric literal at EOF at line 1, column 5" /usr/bin/jq: Argument list too long – Josh Dec 12 '22 at 21:59
@Josh single quotes prevent parameter expansion by the shell. What is the content of the variable? Can you printf/echo it without jq? – knittl Dec 12 '22 at 22:04
It's a large json string (the result of GET /repos/{org}/{repo}/actions/runs) If I change the first cmd line to echo '${{ steps.get_in_progress_workflow_run.outputs.data }}'; then the echo works But if I change it to the same thing with double quotes it fails with a syntax error – Josh Dec 12 '22 at 22:10
assumedly because the json string contains double quotes all over so wrapping it in double quotes is no go – Josh Dec 12 '22 at 22:12
@Josh that's not how shell variables work. They are not "injected" into your command line, they are replaced. It does not matter if they contain double quotes or not. In fact, you should almost always quote variables when expanding. Please try echoing directly without using a variable. See my updated answer. – knittl Dec 12 '22 at 22:14
As stated, if I have just this command echo "${{ steps.get_in_progress_workflow_run.outputs.data }}"; it fails but it works with single quotes. That is echoing directly right? – Josh Dec 12 '22 at 22:15
@Josh you are confusing multiple things. Yes, `echo "${{ steps.... }}"` will fail, because this is a template that's being rendered _before_ being passed to the shell. `x='${{ steps... }'; echo "$x"` requires the double quotes around x and is expanded by the shell. However, please see the last revision of my answer. I think I finally understood what you are trying to do. – knittl Dec 12 '22 at 22:18
Last revision is working, thanks! I did actually think to try that, but I backed off because I thought the double quotes around the command substitution would interfere with the quotes around "in_progress" – Josh Dec 12 '22 at 22:20
You could also try to pass the content in a environment variable. That can then safely be passed along. – jessehouwing Dec 13 '22 at 06:58

Github Action - After using jq to extract a json value the following bash commands still attempt to execute inside jq

1 Answers1