Using AzureFileCopy task for selected files

Question

I have a pretty simple pipeline in Azure DevOps. When a commit is made to a development branch, the files in the repo are checked out and then pushed to a blob storage container using the AzureFileCopy task. When I currently run the pipeline all the files in blob show a modified date including ones which were already in the repo.

Our devs have asked if we can change it so only the new or modified files commited to the repo are updated and all other files are not overwritten. I have tried using the overwrite argument set to false but this ignores any changes to the contents of the files.

I have considered using powershell instead but looking for suggestions on the best way to do this?

could you share you example ; how to setup yaml pipeline. I have similar task on hand. Agent pool running on self hosted VM. Mine pipeline is extact copy of -> https://github.com/MicrosoftDocs/azure-devops-docs/blob/main/docs/pipelines/tasks/includes/yaml/AzureFileCopyV3.md#:~:text=%23%20Example%3A%20Upload,blobPrefix%3A%20targetdirectoryincontainer — khmub, Jan 31 '23 at 06:00

Edward Han-MSFT · Accepted Answer · 2021-04-08T06:24:02.127

You could use Azure PowerShell task to run your script to find the new or modified files committed to the repo and then copy them to your blob storage container. The following is the code snippet.

#get a list of all files that are part of the commit given SHA
$result=$(git diff-tree --no-commit-id --name-status -r $(Build.SourceVersion)) 

#The result looks like "M   test/hello.txt A    today.txt"
$array=$Result.Split(" ") 

#The arraylooks like "
#M   test/hello.txt 
#A   today.txt"
foreach ($ele in $array)
{
    #Added (A), Copied (C), Deleted (D), Modified (M)
    if ($ele.Contains("M") -eq 0 -Or $ele.Contains("A") -eq 0)
    {
        #filename looks like "test/hello.txt"
        $fileName=$ele.Substring(2)
        $sourcePath="$(Build.SourcesDirectory)" + "\" + $fileName

        #your azcopy code here
        
    }
}

Another easier workaround is that you could use the PowerShell task to find the new or modified files committed to the repo and then copy them to a new $(Build.SourcesDirectory)/temp folder with corresponding path structure, and then still use the Azure File Copy task to copy files under the $(Build.SourcesDirectory)/temp folder to your blob storage container.

# Write your PowerShell commands here.

Write-Host "Hello World"

$result=$(git diff-tree --no-commit-id --name-status -r $(Build.SourceVersion))

$array=$Result.Split(" ")

md $(Build.SourcesDirectory)/temp

foreach ($ele in $array)
{
    if ($ele.Contains("M") -eq 0 -Or $ele.Contains("A") -eq 0)
    {
        $fileName=$ele.Substring(2)
        $source="$(Build.SourcesDirectory)" + "\" + $fileName

        $destination="$(Build.SourcesDirectory)/temp" + "\" + $fileName
        
        New-Item $destination -type file -Force

        Copy-Item -Path $source -Destination $destination
    }
}

#Get-ChildItem -Path $(Build.SourcesDirectory)/temp –Recurse

Thanks Edward, that's really helpful, got it working using the second method. :) — JimmyC, Apr 08 '21 at 11:09

Using AzureFileCopy task for selected files

1 Answers1