Using Dockerfile to build custom image for testing on Concourse CI

Question

Due to some requirement to run the tests in some specific test environment we want to build a custom container to run test on Concourse CI.

We are aware of a workflow which is

Concourse --> Build Image --> Push to Artifactory/Docker Hub --> Pull into pipeline --> run container --> run the tests

But we have no requirement to push the image to Hub and would rather prefer a workflow like this

Concourse -- Build Image -- run container -- run the tests

I have checked out out Pivotal's blog, issue but it explains the first workflow.

Can we achieve 2nd workflow ?

Answer 1

You can achieve what you want using the image option when defining your task.

The image option here expects a rootfs and not a docker image though so that is what you will need to provide.

I understand this may seem annoying and counter-intuitive, but it fits in with the concourse abstraction. Docker images are resources, and so they, like all other concourse resources, need to be stored and versioned on some store outside of concourse. Concourse will only download a new image if it finds that a newer one has been uploaded however, so I would argue the correct workflow is a bit different.

What I would do is have a job that feeds in everything you need to a docker-image-resource put, and triggers whenever one of those resources changes. Then in the task that you need the image, pull from your docker store. Concourse will cache the image until it changes, so most of the time you will not need to use up network traffic.

As an example you can see how the Concourse team does this... We have a pipeline that builds images using the docker-image-resource, and then our main pipeline consumes these images, and uses a cached version the vast majority of the time.