Pass build directory (/dist) from a job to next job in concourse

Question

I know its not quite simple to do this and tried to explore many approaches but either I couldn't understand it properly or didn't work for me.

I have a concourse job which runs angular build (ng build) and creates /dist folder. This works well.

jobs:
  - name: cache
    plan:
      - get: source
        trigger: true
      - get: npm-cache
  - name: build
    plan:
      - get: source
        trigger: true
        passed: [cache]
      - get: npm-cache
        passed: [cache]
      - task: run build
        file: source/ci/build.yml

build.yml

---
platform: linux
image_resource:
  type: docker-image
  source: { repository: alexsuch/angular-cli, tag: '7.3' }
inputs:
  - name: source
  - name: npm-cache
    path: /cache
outputs:
  - name: artifact
run:
  path: source/ci/build.sh

build.sh

#!/bin/sh

mv cache/node_modules source

cd source

npm rebuild node-saas # temporary fix

npm run build_prod

cp -R dist ../artifact/

I have mentioned output as artifact where I am storing the dist content. But when I am trying to use this in next job, it doesn't work. Failed with missing input error.

Here is the next job that supposed to consume this dist folder:

jobs:
...
...
  - name: list
    plan:
      - get: npm-cache
        passed: [cache, test, build]
        trigger: true
      - task: list-files
        config:
          platform: linux
          image_resource:
            type: registry-image
            source: { repository: busybox }
          inputs:
          - name: artifact
          run:
            path: ls
            args: ['-la', 'artifact/']

Can anyone please help me with this. How I can use the dist folder in above job.

Peter Arboleda · Answer 1 · 2020-02-26T21:27:55.050

I'm not quite sure why would you want to have different plan definitions for each task but here is the simplest way of doing what you want to do:

jobs:
  - name: deploying-my-app
    plan:
      - get: source
        trigger: true
        passed: []
      - get: npm-cache
        passed: []
      - task: run build
        file: source/ci/build.yml
      - task: list-files
        file: source/ci/list-files.yml

build.yml

---
platform: linux
image_resource:
  type: docker-image
  source: { repository: alexsuch/angular-cli, tag: '7.3' }
inputs:
  - name: source
  - name: npm-cache
    path: /cache
outputs:
  - name: artifact
run:
  path: source/ci/build.sh

list-files.yml

---
platform: linux
image_resource:
  type: registry-image
  source: { repository: busybox }
inputs:
- name: artifact
run:
  path: ls
  args: ['-la', 'artifact/']

build.sh

#!/bin/sh

mv cache/node_modules source
cd source
npm rebuild node-saas # temporary fix
npm run build_prod
cp -R dist ../artifact/

Typically you would pass folders as inputs and outputs between TASKS instead of JOBS (althought there's some alternatives)

Concourse is statelessness and that is the idea behind it. But if you want to pass something between jobs the only way to do that is to use a concourse resource and depending on the nature of the project that could be anything from a git repo to a s3 bucket, docker image etc. You can create your own custom resources too.

Using something like s3 concourse resource for example

This way you can push your artifact to an external storage and then use it again on the next jobs on the get step as a resource. But that just may create some unnecesary complexity understanding that what you want to do is pretty straightforward

In my experience I found that sometimes the visual aspect of a job plan in the concourse dashboard gives the impression that a job-plan should by task atomic, which is not always needed

Hope that helps.

Pass build directory (/dist) from a job to next job in concourse

build.yml

build.sh

1 Answers1