Azure Pipelines: How to do successful BITS-Transfer in Powershell task?

Question

Can't find the answer for this anywhere.

I need my pipeline to call a powershell script that has to do a BITS-Transfer multiple times. At first the script would always exit with this error:

Start-BitsTransfer : The operation being requested was not performed because the user has not 
logged on to the 
network. The specified service does not exist. (Exception from HRESULT: 0x800704DD)

Then, in the Powershell task, instead of calling the file, I changed it to do this inline script:

powershell -File $(Build.Repository.LocalPath)\installer\assembly.win10.x64.ps1 -Credential Get-Credential [Domain]\[Username]

Now the script runs, but I get this in my log in Azure Pipelines:

Microsoft Visual C++ 2015 Redistributable x64 : downloading...
Start-BitsTransfer : The operation being requested was not performed because the user has not 
logged on to the 
network. The specified service does not exist. (Exception from HRESULT: 0x800704DD)
At C:\agents\_work\3\s\installer\assembly.win10.x64.ps1:9 char:5
+     Start-BitsTransfer -Source $Url -Destination $Target
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Start-BitsTransfer], COMException
+ FullyQualifiedErrorId : 
System.Runtime.InteropServices.COMException,Microsoft.BackgroundIntelligentTransfer.Management.NewBitsTransferCommand

Microsoft Visual C++ 2015 Redistributable x64 : download completed

The pipeline passes, but when I check the actual redistributable's .exe, it has a size of 0 bytes. So, it seems the folders are made and zipped, and the files are where they need to be, but the download didn't actually work. I also tested the powershell script on my machine locally, and there are no problems. How can I fix this?

Here is the relevant code. First three blocks are the download-related methods, then lastly is the method call:

function Get-FileSha256([string]$Src) {
  return (Get-FileHash "$Src" -Algorithm SHA256).Hash
}

function Download-File([string]$Comment, [string]$Url, [string]$Target) {
  Write-Host "$Comment : downloading..."
  New-Item -ItemType File -Path "$Target" -Force | Out-Null
  Import-Module BitsTransfer
  Start-BitsTransfer -Source $Url -Destination $Target
  Write-Host "$Comment : download completed"
}

function Download-FileIfNecessary([string]$Comment, [string]$Url, 
  [string]$Target, [string]$Sha256) {
  If (!(Test-Path $Target)) {
    Download-File "$Comment" "$Url" "$Target"
    return
  }

  $TargetSha256 = Get-FileSha256 $Target
  If ($TargetSha256 -eq $Sha256) {
    Write-Host "$Comment : already downloaded"
  }
  Else {
    Download-File "$Comment" "$Url" "$Target"
  }
}

Download-FileIfNecessary "Microsoft Visual C++ 2015 Redistributable x64" `
  "https://download.microsoft.com/download/6/A/A/6AA4EDFF-645B-48C5-81CC-ED5963AEAD48/vc_redist.x64.exe" `
  "$target_downloaded_vc_redist_dir\2015.v14.0.24215\vc_redist.x64.exe" `
  "da66717784c192f1004e856bbcf7b3e13b7bf3ea45932c48e4c9b9a50ca80965"

Also, your second attempt "works" because it's still failing, but the `powershell.exe` isn't exiting with a non-zero exit code and so the pipleline thinks it succeeded. — codewario, Feb 03 '22 at 20:47
I'm also not having issues with your code. Even without the hash check it downloads fine. I fixed your sample as there was whitespace inserted strangely, breaking copy-paste execution of it. For the record, avoid backticks for multiline commands and in code samples. One last question, what happens when you remote into the box through RDP and run the command? — codewario, Feb 03 '22 at 21:15
No issues. It is worth noting that the only account I have the login info for on that machine is an admin account. So I do not need to run an elevated prompt or use "Get-Credential", I can right-click, "Run with Powershell" and it works. I never figured the code was the issue. — Unstable Andy, Feb 03 '22 at 21:23
Does the [solution here work](https://stackoverflow.com/a/22470539/584676) for you? Seems that BITS doesn't like to work if the user who scheduled it isn't logged on interactively (PSRemoting doesn't help here btw). That answer suggests creating a network mount (they use `net use` but I'm not sure if `New-PSDrive`) would work for this or not), running your BITS transfer, and then removing the mount. The local C$ mount seems to satisfy the BITS "logged on" requirement. — codewario, Feb 03 '22 at 21:24
@BendertheGreatest it's no longer failing, but it is hung up after generating the script. Some parts of the script rely on certain relative paths and environment vars, so perhaps I should call it as a file instead of running the whole script inline. Will update this with either another comment or picking a post as answer. — Unstable Andy, Feb 03 '22 at 21:50
I would still recommend not using BITS with Azure Pipeline code. BITS is great for workstations or when running from a Scheduled Task, and it performs well when uninterrupted, but it shouldn't be used during critical processes. The OS can suspend the task at any time for its own uses, such as checking for updates or downloading updates. — codewario, Feb 03 '22 at 23:02

codewario · Accepted Answer · 2022-02-05T20:13:00.433

I'm going to offer an alternative to using BITS, as BITS is designed to be used when the requestor is a local authenticated user, not by remote sessions, service accounts, or other non-interactive contexts. I've re-written Download-File to make use of Invoke-WebRequest instead of
Start-BitsTransfer:

Function Download-File {
  Param(
    [string]$Comment,
    [string]$Url,
    [string]$Target
  )

  Write-Host "$Comment : downloading..."

  # We need to check that the target folder exists later
  $targetFolder = Split-Path -Parent $Target

  # Prepare cmdlet arguments for splatting
  $iwrArgs = @{
    UseBasicParsing = $true
    Uri             = $Url
    OutFile         = $Target
  }

  $newDirArgs = @{
    ItemType = 'Directory'
    Path     = $targetFolder
  }

  # Create the target parent dir, if it doesn't exist
  if ( !( Test-Path -PathType Container $targetFolder ) ) {

    Write-Host "Creating directory $targetFolder"
    New-Item @newDirArgs | Out-Null
  }

  # Workaround progress bar performance bug with Invoke-WebRequest
  # (also affects Invoke-RestMethod) by disabling it
  $OldProgressPreference = $ProgressPreference
  $ProgressPreference = 'SilentlyContinue'

  # Use a try / finally block to guarantee the
  # original progress preference is changed back
  try {

    Invoke-WebRequest @iwrArgs -EA Stop
  }
  finally {

    $ProgressPreference = $OldProgressPreference
  }
}

I've commented the code if you're interested in what each piece is doing. I can clarify if there are any questions.

You should be able to drop this in place of your current Download-File definition. I would recommend this over trying hacky OS tricks to make BITS work in a context it's not designed to. BITS is also not recommended for use with critical automation, as it downloads can be suspended by the OS at any time (e.g. when updates are being downloaded).

Note that what I've said is true for non-interactive logins. BITS should be safe to use from a scheduled task or user-run script, just not from a Windows service (which the Azure Agent is).

Azure Pipelines: How to do successful BITS-Transfer in Powershell task?

1 Answers1