3

I've been working with CircleCI, TravisCI and Azure DevOps for quite some time now.

While it's great to have a managed server that integrates beautifully with all of the external services you can think of (VCS, CD pipeline, app stores etc.), one of my greatest pains is testing for CI. When setting up such systems, I spend the majority of my days tweaking and configuring YAML files, Bash scripts and other parts of the CI process to work faster and smarter. However, this process is slow: it takes 1-5 minutes to even get the first error message that something is wrong, not to mention sometimes 1+ hours for final builds.

You can only do so much locally - mock environment variables, use exactly the right container and add credentials for the relevant services. But it's not quite there, just yet. I envision a full-service solution for this, one that takes care of setting up the environment exactly the same as in Azure or Travis, including getting data from Git based on the specific push and the like.

I would be able to pay good money for that - this will save me hours every week of waiting for the CI agents to spin and finish small ls, pwd and echo statements just to figure out where the heck I am.

Is there something like that out there?

Edit: This is a good example of a "duck-tape" solution that could be of great use, but is not full-fledged. I'm looking for something like this, but more robust.

Edit 2: This is a bad example of a solution that requires me to do the most crucial part - dependencies - manually.

Edit 3: This has a lot of interesting options, specifically using Chef and travis-cookbooks or vagrant. This is the most promising direction so far, albeit it's for Travis alone, seemingly. Maybe set up vagrant boxes for each type of agent you have?

riQQ
  • 9,878
  • 7
  • 49
  • 66
Tom Granot
  • 1,840
  • 4
  • 22
  • 52
  • I am not sure I understand the question - why would you want to emulate a CI environment that was not actually CI? Are you wanting to determine whether a specific build would pass by running it locally, without actually pushing it? – halfer Jun 07 '19 at 22:02
  • Since I spend the majority of my days tweaking and configuring YAML files, bash scripts and other parts of the CI process to work faster and smarter. Think how slower a programmer is if it takes them 1-5 minutes to even get the first error message that something is wrong, not to mention sometimes 1+ hours for final builds.... – Tom Granot Jun 11 '19 at 18:51
  • This is a great question! It certainly should not be closed. Every time I make changes to CI it's a case of making a change, triggering off the CI and waiting. Best practice says we should make small changes but the process here makes you want to do several changes at once to speed things up and then when things break you have to unpick it all. Testing CI currently feels like going back 20 years in terms of testing. – Matt Jun 12 '19 at 06:56
  • 1
    @halfer - this isn't about whether a simple build will work, it's whether the entire CI process works. There could be several builds, several sets of test, artifacts being copied and published, tokens being replaced, etc. etc. – Matt Jun 12 '19 at 07:00
  • @t0mgs So you want to make your CI faster? I don't fully understand the question. – Shayki Abramczyk Jun 13 '19 at 09:08
  • @ShaykiAbramczyk I want to make **TESTING** for my CI faster. If I'm checking stuff on `azure-pipelines.yml`, and doing small tweaks - not large stuff - like checking values of environment variables at certain build phases, I don't want to spin up the CI server. I just want to see what is going on, and move to the next tweak. – Tom Granot Jun 13 '19 at 10:48
  • @t0mgs So, assuming you have a big `azure-pipelines.yml` and you want to run only 1 or 2 steps there (to test things)? – Shayki Abramczyk Jun 13 '19 at 11:00
  • @ShaykiAbramczyk That would be one example, which I'm currently using [conditions](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml) and some tricks with commit messages to get around. – Tom Granot Jun 13 '19 at 11:02
  • @t0mgs I also thought about custom conditions, you can put on each step except the tweaks a condition to run only if `x` not equal to `y`, and when you want to test you set `x` to be with value `y`. – Shayki Abramczyk Jun 13 '19 at 11:07
  • I could, and I do. But I'm looking for something **local**, that does not occupy running agents, does not break `master`, basically does not affect anyone but me and is **local** (twice, just to make sure the point comes across:)) – Tom Granot Jun 13 '19 at 14:53
  • Possibly worth looking into using local build agents to test on, there’s a [docker image](https://hub.docker.com/_/microsoft-azure-pipelines-vsts-agent) for them – Matt Jun 13 '19 at 15:07
  • Using them, but for actual builds. I'm thinking of setting a separate pool just for testing. This won't work for Travis and Circle and Codeship and what have you though. It's local to Azure. – Tom Granot Jun 13 '19 at 15:08
  • 1
    @t0mgs - you might want to follow this github issue: https://github.com/microsoft/azure-pipelines-yaml/issues/32 – Matt Jun 27 '19 at 21:40
  • @Matt Will do, thank you!! – Tom Granot Aug 27 '19 at 19:08

0 Answers0