24

I am using a self-hosted agent for running my build and release pipelines.

A problem happens after I run a build and it fails due to an issue in the pipeline. The cloned branch from remote sits in the working directory. The second run also uses the same working directory, not a new one. I have verified on the agent that no new directory was created. I can also see in the Azure Pipeline logs that it's using the same old directory.

What I did temporarily to solve this was I located the old working directory (e.g. /home/user/_work/13) and manually deleted it. Then Azure creates a new folder (e.g. "14") under _work, and I could see the latest cloned code from the remote in that new directory.

How can I automatically delete the working folder (the numbered directory under _work) when a failure occurs?

Note: I already have put clean-up steps in place at the end of my pipeline which run when the pipeline completes with success. Also, I am writing YAML pipelines, not Classic.

Please let me know if any information is required for a better understanding.

bubbleking
  • 3,329
  • 3
  • 29
  • 49
Indrajeet Gour
  • 4,020
  • 5
  • 43
  • 70
  • 1
    Are you using Git or TFVC? Are you using YAML pipelines or classic pipelines? Did you look at the documentation for working folder cleaning options? – Daniel Mann Jul 01 '20 at 17:05
  • I am using the azure git repository, I am not getting why someone down voted this, if they found it silly why do not they answer it .. – Indrajeet Gour Jul 01 '20 at 18:26
  • @DanielMann, let me know if anything else you required for better understanding. thank you – Indrajeet Gour Jul 01 '20 at 19:21

2 Answers2

47

Azure DevOps - Clean build directory

There is a Clean option on the Get Sources tab, which can perform different kinds of cleaning of the working directory of your private agent before the build is run:

enter image description here

We could set the value to true to clean the working directory of your private agent. Even if the build is failed.

You could check the document Clean the local repo on the agent for some more details.

Update:

But this is meant for the classic pipeline, do not we have any tag which we define in yml pipeline only

jobs:
- job: string  # name of the job (A-Z, a-z, 0-9, and underscore)
  ...
  workspace:
    clean: outputs | resources | all # what to clean up before the job runs

Check this document YAML schema reference for some details. Hope this helps.

Mark Wragg
  • 22,105
  • 7
  • 39
  • 68
Leo Liu
  • 71,098
  • 10
  • 114
  • 135
  • 1
    Thank you for a response, but this is for the classic pipeline, do not we have any tag which we define in yml pipeline only – Indrajeet Gour Jul 02 '20 at 07:57
  • @YouAreAwesome, hm, you didn’t mention a bit about YAML in your original question, so I thought you were in classic mode. Now, I have updated the answer for your comment, please check if it help you. – Leo Liu Jul 02 '20 at 08:04
  • 1
    I missed from the document somehow, thank you for the reference link appreciated. – Indrajeet Gour Jul 02 '20 at 08:15
  • 1
    I could not find any information about cleaning up the workspace directory in the YAML schema reference link provided in your answer. However you did point me in the right direction and I found on google the following Azdo page describing how to do it: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#workspace – guicey Jun 07 '22 at 14:09
  • How do we clean after the job? – Verthais Sep 29 '22 at 08:15
  • This option is not present on release pipelines... – Gert van den Berg May 23 '23 at 09:30
0

Since running pre-cleanup-jobs (workspace.clean) or post-cleanup-jobs some step X to cleanup the agent workdir heavily relies on the pipeline-author of "some pipeline" to fix a potential hard to detect "workdir pollution" issue, we decided to re-implement the idea to guarantee workdir pollution can never happen.

I have created https://github.com/EugenMayer/azure-agent-self-hosted-toolkit which fixes this project on the agent-level, not requiring any changes to the pipelines nor relying on those.

To quote to project idea

The run-once mode is based on Microsoft's `./run.sh --once` which ensures that an agents only runs 1 job and then stops.
This is used to

 - cleanup the workdir in a safe manner after each job
 - ensures each job on an agent runs in a clean workdir
 - starts the agent right after cleanup up (few seconds) to be available for the next job

This repository also offers a toolkit to start / setup x-agents and maintain them using the original tools of Microsoft, but wrapped in convenient scripts.

If this helps anybody else, happy to share it.

Eugen Mayer
  • 8,942
  • 4
  • 33
  • 57