VSTS Hosted Agent, not enough space in the disk

Question

I cannot build in VSTS with hosted agent (VS 2017) with error:

System.IO.IOException: There is not enough space on the disk

I have tried setting "Clean" option to true on Build , Repository definition without solving the issue. I didn't have this option set to true which I imagine led to the current situation.

Also installed VSTS extension "Clean Agent Directories" and added as last step of the build process without solving the issue either.

Is there an option that would allow me to solve this issue and continue using the hosted build agent ?

But how big is the **entire repository**? And are you using Git, or TFVC? — Daniel Mann, Sep 10 '17 at 15:09
Is Git repository. The .git folder inside the solution's local folder is 692MB, if I download the whole repo from VSTS as zip is 19MB and unzipped is 46MB. Sorry if i'm not precise, I'm relatively new to git and VSTS — donquijote, Sep 10 '17 at 15:20
Also in case that helps to understand how the build might multiply the size, the solution is a ASP.NET MVC with multiple webjobs. — donquijote, Sep 10 '17 at 15:26
You can also set up a private agent, and queue your build on it https://www.visualstudio.com/en-us/docs/build/actions/agents/v2-windows. — Marina Liu, Sep 11 '17 at 08:22

score 8 · Accepted Answer · answered Sep 10 '17 at 15:19

Hosted agents offer 10 GB of space. You stated that your entire solution folder is 2.6 GB. Your build outputs will typically be somewhere in the range of 2x that size, if not larger, depending on various factors.

If you're a Git user, this the entire repo that's being cloned may be significantly larger than 2.6 GB, as well -- cloning the repo brings down not only the current working copy of the code, but also all of the history.

You can control the clone depth (e.g. how much history is pulled down) by enabling Shallow fetch under the Advanced options of your repo settings.

If you're a TFVC user, you can check your workspace mappings to ensure only relevant source code is being pulled down.

You may be in a situation where the 10 GB simply isn't sufficient for your purposes. If the 2.6 GB is purely code and contains no binary assets (images, PDFs, video files, etc), you may want to start modularizing your application so smaller subsections can be built and independently deployed. If the 2.6 GB contains a lot of binary assets, you'll likely want to separate static content (images, et al) from source code and devise a separate static content deployment process.

I restructured the solution go have each build-deploy under 1 GB. Nowadays somewhat easy for things to reach few GB. It was good to learn the hosted agent does start clean with fresh 10GB each time if I understood correctly so no necessary to setup additional cleaning steps on the hosted agent. — donquijote, Sep 12 '17 at 09:18
You can also split up your build in multiple agent jobs, things like restore you might have to re-do but it helps reducing the space per agent job. — CularBytes, Apr 09 '21 at 07:18

score 2 · Answer 2 · answered May 22 '19 at 13:32

According to Microsoft's documentation,

(Microsoft-hosted agents) Provide at least 10 GB of storage for your source and build outputs.

So, if you are getting "not enough space in disk error" it might mean that the amount of disk space used by your source code (files, repos, branches, etc), together with the amount of disk space taken by your build output (files generated as a result of the build process) is crossing the 10 GB of storaged provided by your DevOps plan.

When getting this error I had to delete an old git repo and an old git branch, getting 17 MB of free space, which was enough for my build to process. Thus, in my case the space was being used up by source code. It could well be too many or too big files being generated by the build. That is, you just need to find out which one of these two is the cause of your lack of disk space, and work on freeing it.

score 1 · Answer 3 · answered Nov 16 '21 at 14:58

There is a trick to free agent space by removing the cached docker images (if you don't need them of course). With the Microsoft hosted agent there is a list of docker images pre-provisioned. This SO answer describes where to find the docs on the different images / cached container images.

It's as simple as adding an extra command task to cleanup the cached images. For Linux / Ubuntu:

steps:
- script: |
  df -h
- script: |
    docker rmi -f $(docker images -aq)
- script: |
    df -h

The df (disk-free) command shows you how much is saved. This will probably free up another 5Gb.

score 1 · Answer 4 · answered Mar 20 '23 at 15:48

1

For now, I'm freeing up ~ 20gb with this insanity:

df -h
echo now freeing disk space
sudo rm -rf /usr/local/lib/android # android sdks
sudo rm -rf /usr/local/.ghcup # haskell stuff
sudo rm -rf /opt/hostedtoolcache/CodeQL # github code analysis engine
echo freed up space:
df -h

Mostly coming from the multitude of android sdk versions.

answered Mar 20 '23 at 15:48

Dave Higgins

155
1
1
8

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Mar 25 '23 at 00:16
I think its pretty clear if emotive. – Dave Higgins Mar 27 '23 at 08:19

VSTS Hosted Agent, not enough space in the disk

4 Answers4