0

I have a single repo that is segmented into a number of projects, each of which have a build that generates a Docker image. I've configured Azure devops to trigger on the proper paths for each project such that I get the right images build when their subdirectories change. One PR can trigger the build of one or multiple images.

Once all my images are built, I want to trigger an integration test job that starts all of the images that comprise my service. If the PR didn't change one it should run service:latest rather than the one(s) which were built. How can I create a test pipeline that doesn't start until each of the other pipelines finish and know which image version to use? For the image, I think I can just use something like the PR number or commit hash in the image tag and revert to service:latest if that image doesn't exist.

What's not clear is how to have the integration test job wait for all of the dependent (ideally parallelized) builds to finish. I considered the idea of a single pipeline with different build jobs, but this creates another problem of wanting to have independently triggered CDs for each build artifact output. I never want to re-deploy images that haven't changed and I want to be able to track the deployments of those which have changed so a single CD for all builds is a bad idea.

Dave Brennan
  • 68
  • 1
  • 8

1 Answers1

0

You can create a Release pipeline to run your test job and configure invoke rest api Gate to check if all the pipeline were completed. See below steps:

1, Create a release pipeline.

2, Add all the artifacts from the other pipelines as the release pipeline Artifacts source. So that the pipeline will be triggered when there is any new version of artifact being published.

3,Create a generic service connection to connect to your azure devops organization. See this thread for detailed steps.

4, Configure invoke rest api Gate for the release pipeline. See below screenshot.

enter image description here

5, Configure the invoke rest api gate to call Build-latest rest api to get the latest build of the other dependency pipeline. See below screen:

enter image description here

6, Add multiple invoke rest api gate one for each of your dependency pipelines

7, Configure the evaluation option for the Gate.

enter image description here

After above configurations, the release pipeline will be executed if all the pipeline is evaluated to be completed and succeeded.

To decide which image version to use, you can clone your source code in the release pipeline and run below git command to check which folders are changed. Then you can decide which images are to use. Please see this thread for more information.

git diff HEAD HEAD~ --name-only

Levi Lu-MSFT
  • 27,483
  • 2
  • 31
  • 43
  • Our Github repo also depends on the current CI pipelines passing ("Require status checks to pass" option) before a PR can be merged and would require this integration test to pass. I see the advantage of using the gates, but it is not clear how/if to make this work for releases since they don't show up in the status check list. – Dave Brennan Feb 05 '21 at 17:23
  • And there's another potential problem here, a PR may only trigger a rebuild of n <= N total artifacts that are used by the test. What if only 1 of N rebuilds, how will the gate definitions know to wait only for the ones that were necessary to rebuild for this particular test? The gate definitions suggested above will check on the latest build for each pipeline which is not correct in this case and also not correct in a multi-user system which may have many builds in each pipeline running at once. They have to be tied together somehow by PR branch or PR number I would think. – Dave Brennan Feb 05 '21 at 19:52