Github Actions: Why an intermediate command failure in shell script would cause the whole step to fail?

Question

I have a step in a Github Actions job:

      - name: Check for changes
        run: |
          diff=$( git diff --name-only 'origin/main' )
          changed_files=$( echo $diff | grep -c src/my_folder ) # this fails

          # more script lines
          # that are unrelated

This fails with Error: Process completed with exit code 1. only if grep finds nothing. If there are matches in $diff, then this step works as intended. But of course it also needs to work without matches.

I can run this locally or inside a script without a problem, exit code is always 0 (on a Mac).

I fail to understand what the problem is. After some hours of trial and error and research I learned that apparently grep is tricky in Github actions, but I found no hint or proper documentation how I am supposed to solve this exact case.

If I change my failing line to

echo $( echo $diff | grep -c src/my_folder ) # this works and prints out the result

this gets executed without problems.

But how do I get my grep output into my variable even when there are no findings?

sorry, a bit off topic, why not using an existing action in order to filter for file changes, like [dorny/paths-filter](https://github.com/dorny/paths-filter)? — Matteo, Jul 21 '22 at 12:56

pynexj · Accepted Answer · 2023-02-11T10:59:05.783

9

According to the doc, by default Github Actions enables set -e to run the step's commands. This is why an intermediate command failure may cause the whole step to fail. To take full control of your step's commands you can do like this:

  - name: Check for changes
    shell: bash {0}
    run: |
      diff=$( git diff --name-only 'origin/main' )
      changed_files=$( echo $diff | grep -c src/my_folder )

      # ...
      # ...

Or you can just disable the default set -e at the beginning of the step's script:

  - name: Check for changes
    run: |
      set +x

      diff=$( git diff --name-only 'origin/main' )
      changed_files=$( echo $diff | grep -c src/my_folder )

      # ...
      # ...

edited Feb 11 '23 at 10:59

answered Jul 21 '22 at 16:26

pynexj

19,215
5
38
56

This solved my problem, and is a good thing to know! I didn't figure out why my command behaves different when run by Github Actions (yet), but with `shell: bash {0}` my shell doesn't stumble over it anymore. Thank you! – michpohl Jul 28 '22 at 06:11
glad it helped. i myself prefer the `bash {0}` way. no surprise any more. :) – pynexj Jul 28 '22 at 06:28
I was wondering if it is actually safe to rely on GitHub using `set -e` semantics automatically. I was trying to find something on it in the documentation. What I found actually seems to state the opposite (`set -e` is not applied by default), which is odd. I'm trying to clarify that in a [separate question](https://stackoverflow.com/q/75419587/1804173). – bluenote10 Feb 11 '23 at 10:43
@bluenote10 see the doc: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun – pynexj Feb 11 '23 at 10:59

score 0 · Answer 2 · answered Jul 21 '22 at 12:59

0

I would suggest to use the dorny/paths-filter action, like:

- uses: dorny/paths-filter@v2
  id: changes
  with:
    filters: |
      src:
        - 'src/my_folder/**'

  # run only if some file in 'src/my_folder' folder was changed
- if: steps.changes.outputs.src == 'true'
  run: ...

Check the output section for details about changes

answered Jul 21 '22 at 12:59

Matteo

37,680
11
100
115

1

This might be a last resort. I prefer to keep things as simple as I can, which is why I prefer bash. This way everything is under my control and there is as little overhead as possible. So, while this might solve the problem, I'd still be interested in how to get `grep` to work as intended... – michpohl Jul 21 '22 at 13:32

score 0 · Answer 3 · answered Jul 21 '22 at 14:20

I don't know what problem grep has in Github actions but you can try something like :

... 
changed_files=$( [ ! -e $diff ] && { echo $diff | grep -c src/my_folder } || echo 0 ) 
...

This way grep wouldn't run if $diff is empty. It would just store 0 in $changed_files

Github Actions: Why an intermediate command failure in shell script would cause the whole step to fail?

3 Answers3

Linked