Using output from a previous job in a new one in a GitHub Action

Question

For (mainly) pedagogical reasons, I'm trying to run this workflow in GitHub actions:

name: "We  Perl"
on:
  issues:
    types: [opened, edited, milestoned]

jobs:
  seasonal_greetings:
    runs-on: windows-latest
    steps:
      - name: Maybe greet
        id: maybe-greet
        env:
          HEY: "Hey you!"
          GREETING: "Merry Xmas to you too!"
          BODY: ${{ github.event.issue.body }}
        run: |
          $output=(perl -e 'print ($ENV{BODY} =~ /Merry/)?$ENV{GREETING}:$ENV{HEY};')
          Write-Output "::set-output name=GREET::$output"
  produce_comment:
    name: Respond to issue
    runs-on: ubuntu-latest
    steps:
      - name: Dump job context
        env:
          JOB_CONTEXT: ${{ jobs.maybe-greet.steps.id }}
        run: echo "$JOB_CONTEXT"

I need two different jobs, since they use different context (operating systems), but I need to get the output of a step in the first job to the second job. I am trying with several combinations of the jobs context as found here but there does not seem to be any way to do that. Apparently, jobs is just the name of a YAML variable that does not really have a context, and the context job contains just the success or failure. Any idea?

VonC · Accepted Answer · 2023-04-04T18:50:27.473

197

Check the "GitHub Actions: New workflow features" from April 2020, which could help in your case (to reference step outputs from previous jobs)

Job outputs

You can specify a set of outputs that you want to pass to subsequent jobs and then access those values from your needs context.

See documentation:

jobs.<jobs_id>.outputs

A map of outputs for a job.

Job outputs are available to all downstream jobs that depend on this job.
For more information on defining job dependencies, see jobs.<job_id>.needs.

Job outputs are strings, and job outputs containing expressions are evaluated on the runner at the end of each job. Outputs containing secrets are redacted on the runner and not sent to GitHub Actions.

To use job outputs in a dependent job, you can use the needs context.
For more information, see "Context and expression syntax for GitHub Actions."

To use job outputs in a dependent job, you can use the needs context.

Example

jobs:
  job1:
    runs-on: ubuntu-latest
    # Map a step output to a job output
    outputs:
      output1: ${{ steps.step1.outputs.test }}
      output2: ${{ steps.step2.outputs.test }}
    steps:
    - id: step1
      run: echo "test=hello" >> $GITHUB_OUTPUT
    - id: step2
      run: echo "test=world" >> $GITHUB_OUTPUT
  job2:
    runs-on: ubuntu-latest
    needs: job1
    steps:
    - run: echo ${{needs.job1.outputs.output1}} ${{needs.job1.outputs.output2}}

Note the use of $GITHUB_OUTPUT, instead of the older ::set-output now (Oct. 2022) deprecated.

To avoid untrusted logged data to use set-state and set-output workflow commands without the intention of the workflow author we have introduced a new set of environment files to manage state and output.

Jesse Adelman adds in the comments:

This seems to not work well for anything beyond a static string.
How, for example, would I take a multiline text output of step (say, I'm running a pytest or similar) and use that output in another job?

either write the multi-line text to a file (jschmitter's comment)
or multi-line text to >> $GITHUB_OUTPUT (maxisam's answer)
or base64-encode the output and then decode it in the next job (Nate Karasch's comment)

edited Apr 04 '23 at 18:50

answered Apr 15 '20 at 19:14

VonC

1,262,500
529
4,410
5,250

Thanks for this. This seems to not work well for anything beyond a static string. How, for example, would I take a multiline text output of step (say, I'm running a `pytest` or similar) and use that output in another job? – Jesse Adelman Jul 27 '21 at 00:36
2

@JesseAdelman Good question: I am not sure. You should ask a separate question with a link back to this answer, to get a more precise answer. – VonC Jul 27 '21 at 06:24
1

@Jesse Adelman, you'd probably want to write the multi-line text to a file in that case. – jschmitter Sep 30 '21 at 14:35
2

Alternatively, you could base64-encode the output and then decode it in the next job – Nathan Karasch Feb 02 '22 at 18:38
3

For anyone else who is wondering why no output is generated; Check that your steps are using `id` instead of `name` – flymg Feb 15 '22 at 07:43
@VonC, Note the `echo "test=world" >> $GITHUB_OUTPUT` causes ambiguous redirect error, using the old style works: `echo "::set-output name=test::world"` – Mercury Jan 18 '23 at 09:53
@Mercury Considering you should not rely on the deprecated form, how would you make the new one work? – VonC Jan 18 '23 at 11:27
@VonC, I Don't know, I get either `ambiguous redirect` or `No such file or directory` (When i surround the $GITHUB_OUTPUT in quotes) – Mercury Jan 18 '23 at 13:57
@Mercury Then you need to ask a separate question to illustrate what your pipeline contains and ask why it produces this error message. – VonC Jan 18 '23 at 15:04
Here is the way for multiline output https://stackoverflow.com/a/75932020/667767 – maxisam Apr 04 '23 at 17:05
1

@maxisam Well done. I have referenced your answer in mine, for more visibility. – VonC Apr 04 '23 at 18:50

peterevans · Answer 2 · 2021-11-25T23:56:38.220

Update: It's now possible to set job outputs that can be used to transfer string values to downstream jobs. See this answer.

What follows is the original answer. These techniques might still be useful for some use cases.

Write the data to file and use actions/upload-artifact and actions/download-artifact. A bit awkward, but it works.
Create a repository dispatch event and send the data to a second workflow. I prefer this method personally, but the downside is that it needs a repo scoped PAT.

Here is an example of how the second way could work. It uses repository-dispatch action.

name: "We  Perl"
on:
  issues:
    types: [opened, edited, milestoned]

jobs:
  seasonal_greetings:
    runs-on: windows-latest
    steps:
      - name: Maybe greet
        id: maybe-greet
        env:
          HEY: "Hey you!"
          GREETING: "Merry Xmas to you too!"
          BODY: ${{ github.event.issue.body }}
        run: |
          $output=(perl -e 'print ($ENV{BODY} =~ /Merry/)?$ENV{GREETING}:$ENV{HEY};')
          Write-Output "::set-output name=GREET::$output"
      - name: Repository Dispatch
        uses: peter-evans/repository-dispatch@v1
        with:
          token: ${{ secrets.REPO_ACCESS_TOKEN }}
          event-type: my-event
          client-payload: '{"greet": "${{ steps.maybe-greet.outputs.GREET }}"}'

This triggers a repository dispatch workflow in the same repository.

name: Repository Dispatch
on:
  repository_dispatch:
    types: [my-event]
jobs:
  myEvent:
    runs-on: ubuntu-latest
    steps:
      - run: echo ${{ github.event.client_payload.greet }}

Additionally, the second workflow will not appear linked to the original push or pull request. — jjmerelo, Dec 16 '20 at 15:39

EvanK · Answer 3 · 2022-01-28T21:53:24.260

It is possible to capture the entire output (and return code) of a command within a run step, which I've written up here to hopefully save someone else the headache. Fair warning, it requires a lot of shell trickery and a multiline run to ensure everything happens within a single shell instance.

In my case, I needed to invoke a script and capture the entirety of its stdout for use in a later step, as well as preserve its outcome for error checking:

# capture stdout from script 
SCRIPT_OUTPUT=$(./do-something.sh)

# capture exit code as well
SCRIPT_RC=$?

# FYI, this would get stdout AND stderr
SCRIPT_ALL_OUTPUT=$(./do-something.sh 2>&1)

Since Github's job outputs only seem to be able to capture a single line of text, I also had to escape any newlines for the output:

echo "::set-output name=stdout::${SCRIPT_OUTPUT//$'\n'/\\n}"

Additionally, I needed to ultimately return the script's exit code to correctly indicate whether it failed. The whole shebang ends up looking like this:

- name: A run step with stdout as a captured output
  id: myscript
  run: |
    # run in subshell, capturiing stdout to var
    SCRIPT_OUTPUT=$(./do-something.sh)
    # capture exit code too
    SCRIPT_RC=$?
    # print a single line output for github
    echo "::set-output name=stdout::${SCRIPT_OUTPUT//$'\n'/\\n}"
    # exit with the script status
    exit $SCRIPT_RC
  continue-on-error: true
- name: Add above outcome and output as an issue comment
  uses: actions/github-script@v5
  env:
    STEP_OUTPUT: ${{ steps.myscript.outputs.stdout }}
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    script: |
      // indicates whather script succeeded or not
      let comment = `Script finished with \`${{ steps.myscript.outcome }}\`\n`;

      // adds stdout, unescaping newlines again to make it readable
      comment += `<details><summary>Show Output</summary>

      \`\`\`
      ${process.env.STEP_OUTPUT.replace(/\\n/g, '\n')}
      \`\`\`

      </details>`;

      // add the whole damn thing as an issue comment
      github.rest.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner,
        repo: context.repo.repo,
        body: comment
      })

Edit: there is also an action to accomplish this with much less bootstrapping, which I only just found.

score 5 · Answer 4 · edited Jul 03 '22 at 21:27

In my case I wanted to pass an entire build/artifact, not just a string:

name: Build something on Ubuntu then use it on MacOS

on:
  workflow_dispatch:
    # Allows for manual build trigger

jobs:
  buildUbuntuProject:
    name: Builds the project on Ubuntu (Put your stuff here)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: some/compile-action@v99
      - uses: actions/upload-artifact@v2
        # Upload the artifact so the MacOS runner do something with it
        with:
          name: CompiledProject
          path: pathToCompiledProject
  doSomethingOnMacOS:
    name: Runs the program on MacOS or something
    runs-on: macos-latest
    needs: buildUbuntuProject # Needed so the job waits for the Ubuntu job to finish
    steps:
      - uses: actions/download-artifact@master
        with:
          name: CompiledProject
          path: somewhereToPutItOnMacOSRunner
      - run: ls somewhereToPutItOnMacOSRunner # See the artifact on the MacOS runner

score 3 · Answer 5 · answered Oct 13 '22 at 12:37

3

2022 October update: GitHub is deprecating set-output and recommends to use GITHUB_OUTPUT instead. The syntax for defining the outputs and referencing them in other steps, jobs.

An example from the docs:

- name: Set color
  id: random-color-generator
  run: echo "SELECTED_COLOR=green" >> $GITHUB_OUTPUT
- name: Get color
  run: echo "The selected color is ${{ steps.random-color-generator.outputs.SELECTED_COLOR }}"

answered Oct 13 '22 at 12:37

dimisjim

368
2
19

Good point. I have edited [my own answer accordingly](https://stackoverflow.com/a/61236803/6309). – VonC Oct 13 '22 at 12:43
This new method doesn't seem to work with `fromJson()` like the old method did. It complains that the output is a string type instead of the expected JSON. – Christopher Oct 14 '22 at 20:29

score 3 · Answer 6 · answered Apr 04 '23 at 17:03

3

To update @vonc https://stackoverflow.com/a/61236803/667767 on multiline output.

Here is the way to do it

      echo 'RESULT<<EOF' >> $GITHUB_OUTPUT
      echo $multiLine >> $GITHUB_OUTPUT
      echo 'EOF' >> $GITHUB_OUTPUT

answered Apr 04 '23 at 17:03

maxisam

21,975
9
75
84

Using output from a previous job in a new one in a GitHub Action

6 Answers6

Job outputs

Linked

Related