11

I have a .net core 3.1 solution with several web and class library projects in it. All packages use the <PackageReference> format in the .csproj file.

I am building the solution with Azure DevOps Pipelines and I want to reduce my build times by caching Nuget packages instead of restoring them from nuget.org on every run.

Following some guidance from documentation and blog posts, I've:

  1. Added this nuget.config file to my solution so packages are always restored from nuget.org:

    <?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>
</configuration>

  2. Added a Directory.Build.props file to my solution so that for each project, a packages.lock.json will be generated when building.

    <Project>
  <PropertyGroup>
    <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
    <DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>
  </PropertyGroup>
</Project>

  3. Added the generated packages.lock.json files to git.

  4. Integrated the Cache@2 task into my azure-pipeline.yml file like so:

    trigger:
- master
- users/*

pool:
  vmImage: 'windows-latest'

variables:
  buildConfiguration: 'Debug'
  NUGET_PACKAGES: $(Pipeline.Workspace)/.nuget/packages

steps:
- task: Cache@2
  displayName: Cache nuget packages
  inputs:
    key: 'nuget 5 | "$(Agent.OS)" | **/packages.lock.json,!**/bin/**'
    restoreKeys: |
       nuget 5 | "$(Agent.OS)"
    path: $(NUGET_PACKAGES)
    cacheHitVar: CACHE_RESTORED5

- task: DotNetCoreCLI@2
  displayName: 'Restoring nuget packages ($(buildConfiguration))'
  condition: ne(variables.CACHE_RESTORED5, 'true')
  inputs:
    command: 'restore'
    projects: '**/*.csproj'
    includeNuGetOrg: true
    arguments: '--configuration $(buildConfiguration)'

- task: DotNetCoreCLI@2
  displayName: 'Build solution ($(buildConfiguration))'
  inputs:
    command: 'build'
    arguments: '--configuration $(buildConfiguration) --no-restore'

I've managed to run the pipeline successfully so on the first run it restores packages from nuget.org and saves them to the cache and on subsequent runs it reads them from the cache instead.

My problem is that every once in a while my build breaks at the "Build solution" stage with this kind of error message showing for each one of my solution projects:

##[error]C:\Program Files\dotnet\sdk\3.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(241,5): Error NETSDK1004: Assets file 'D:\a\1\s\MyProject\obj\project.assets.json' not found. Run a NuGet package restore to generate this file.

​When this happens I am forced to increment my cache key and environment variable to force a cache refresh and then the following build works.

My question is - Why is this happening and how can I fix it so my build stops being so flaky?

urig
  • 16,016
  • 26
  • 115
  • 184
  • Hi urig, do you have any other concern about this case? – PatrickLu-MSFT Oct 21 '20 at 09:16
  • 2
    @PatrickLu-MSFT thanks. Seeing as I've found no instructions on how to cache the `project.assets.json` files along with the nuget packages, my only issue is that *I can't use the Cache@2 task to cache nuget packages on a Microsoft-hosted agent*. That doesn't make sense to me but I guess I'll just have to forgo caching as I cannot switch to a self-hosted agent at this time. – urig Oct 25 '20 at 13:19
  • 3
    We just saw this problem with a .NET 4.7 project in Azure Pipelines. Workaround is to add a new task in the pipeline to run the command `dotnet restore` after the `nuget restore` command – jmbmage May 23 '22 at 17:56
  • 1
    Had the same issue with a .NET 4.8 Specflow project. The workaround was the same as mentioned by @jmbmage – MarcolinoPT Jun 13 '22 at 15:57

2 Answers2

4

Update:

Caching can be effective at improving build time provided the time to restore and save the cache is less than the time to produce the output again from scratch. Because of this, caching may not be effective in all scenarios and may actually have a negative impact on build time.

Since the project.assets.json needs to be regenerated on the build machine with the resolved paths to this shared cache even if no packages need to be downloaded.

For this scenario, to improve the performance, you can run pipeline on self-hosted agent or push these packages to DevOps repo directly.

More details please take a look at this blog:

According to your description and error info, NUGET_PACKAGES: $(Pipeline.Workspace)/.nuget/packages.

Looks like you didn't cached project.assets.json which caused this issue.

This problem happening when your build tool is not set to do restore on projects set to use PackageReference vs packages.config and mostly affect Net Core and Netstandard new style projects.

When this happens I am forced to increment my cache key and environment variable to force a cache refresh and then the following build works.

Actually it's not related to key, when you force use a newly key. Azure DevOps Service is not using your cached package and force "run a new NuGet Restore". And during the complete restore process, this file generated succeed. Thus build also succeed.

PatrickLu-MSFT
  • 49,478
  • 5
  • 35
  • 62
  • Thanks @PatrickLu-MSFT. What you write makes sense. Can you elaborate on how I can cache the `project.assets.json` file? What changes do I need to make to my `azure-pipelines.yml` file? – urig Oct 15 '20 at 20:07
  • 1
    @urig According to your YAML file, `pool: vmImage: 'windows-latest'` You are using host agent, I am afraid, it's not able to do this. For your scenario(reduce your build time), suggest you still involves a NuGet restore but change to self-host agent to run your pipeline. Sorry for any inconvenience. – PatrickLu-MSFT Oct 16 '20 at 09:40
0

You need to add a task in devops "dotnet restore"

k4st0r42
  • 1,174
  • 1
  • 15
  • 29