NuGetCommand doesn't restore conditional packages

Question

I have a C# project that defines two packages with conditions:

// MyProj.csproj

  <Target Name="OpenCV Version" BeforeTargets="CoreCompile">
    <Message Text="Using NenPack V5" Importance="high" Condition="$(UseVersion5)"/>
    <Message Text="Using NenPack V4" Importance="high" Condition="!$(UseVersion5)"/>
  </Target>

  </ItemGroup>
    <PackageReference Include="NenPack" Version="5.1.7" Condition="$(UseVersion5)" />
    <PackageReference Include="NenPack" Version="4.9.2" Condition="!$(UseVersion5)" />
  </ItemGroup>

The build flag UseVersion5 determines if to load the v5 version of NenPack (private NuGet)

In Azure Dev Ops we have configured a CI pipeline:

// azure-pipeline.yml

    - task: NuGetCommand@2
      displayName: 'Restore Packages for Packing'
      inputs:
        command: 'restore'
        restoreSolution: '**/*.sln'
        feedsToUse: 'config'
        nugetConfigPath: 'nuget.config'
        externalFeedCredentials: 'PrivateFeedCred'
        buildProperties: 'UseVersion5=true'
        noCache: true

    - task: MSBuild@1
      displayName: 'NuGet Packing'
      inputs:
        solution: '**/*.csproj'
        platform: '$(buildPlatform)'
        configuration: '$(buildConfiguration)'
        msbuildArguments: '/noAutoResponse /restore:false /t:Pack /p:UseVersion5=true /p:PackageOutputPath=$(nugetOutput)'
        maximumCpuCount: true

Although the UseVersion5 is set to true and the log output says Using NenPack V5 (so the condition is working), it still use v4 in the packed output nuspec file:

// MyProj.nuspec

<dependency id="NenPack" version="4.9.2" exclude="Build,Analyzers" />

Why is it ignoring the PackageReference condition and take the V4 version instead of V5 although we are asking for 5 and get the right log message?

Any way to enforce the NuGetCommand to download the V5 package?