20

I'm using VSTS as a build server, and while building I want to copy the bin folder contents to the root of the target, and also custom files from another folder to this target. MSDN suggests I use a minimatch pattern, but it's copying files with the subdirectory structure. I'm not interested in restoring the structure.

For example, I am getting this folder structure:

Project
    MyProjectFiles
    bin
        x86 (it's build configuration)
            Project.exe
    Other project files
    Project.sln
SomeScrips
    script1.ps1

But I want to receive this folder structure:

Project.exe
SomeScripts
    script.ps1

Which minimatch pattern can I use for my requirements?

Scott Koland
  • 739
  • 9
  • 18
Georgy Grigoryev
  • 822
  • 2
  • 8
  • 26
  • Are you using XAML or the new web based build system? – Pascal Berger Jan 04 '16 at 09:16
  • new web based system – Georgy Grigoryev Jan 04 '16 at 09:50
  • None of the answers take non-trivial scenarios (more than one project) into account except maybe DotBerts'. File and Folder handling in vNext builds is arguably worse than XAML, especially when you want to get the artifact from $(build.sources)\setup\MyInstaller\bin\Release\myinstaller.msi copied to the $(build.artifactstaging) folder (not $(build.artifactstaging)\setup\MyInstaller\bin\Release\) – StingyJack Jul 25 '17 at 12:01

7 Answers7

28

You need to specify the copy root if you want to copy files only without folder structure. Since the project.exe is in a different path with script.ps1 file, you need to copy them in different copy task.

Following the steps below:

  1. Add a "Copy Files" step to copy "project.exe". Settings like following: enter image description here
  2. Add a "Copy Files" step to copy "SomeScripts" folder. Settings like following: enter image description here
  3. Add a "Copy and Publish Build Artifacts" step to copy these files to "drop". Settings like following: enter image description here

Now you should get the things like following in drop folder:

Project.exe
SomeScripts
    script.ps1
Eddie Chen - MSFT
  • 29,708
  • 2
  • 46
  • 60
  • 1
    This looks like exactly what I want to do so I'm upvoting, but we don't have the Copy Files step available in our on-premise TFS Build instance. Is this another example of how MS decided to cripple on premise installations for no good reason? This is pretty basic functionality to leave out. Poor show MS, poor show. – Steve Pettifer Mar 09 '16 at 14:56
  • 1
    @StevePettifer Sorry for the inconvenience. Compare to on-premise TFS, VSTS is online service and use a different release model, the updates and fixes can be delivered continuously. So the build tasks keeps growing. That's why you didn't see some tasks in TFS. These tasks are available in TFS2015 Update2 RC2 now: https://www.visualstudio.com/en-us/news/tfs2015-update2-vs.aspx. By the way, all VSTS tasks are placed on GitHub, you can download them and upload them into your TFS Server. See this link https://github.com/Microsoft/vso-agent-tasks and this: https://github.com/Microsoft/tfs-cli – Eddie Chen - MSFT Mar 10 '16 at 07:05
  • Thanks, that's good to know. Sadly I can't influence updating TFS (we're big and bureaucratic enough to have dedicated TFS/ALM admins and corporate inertia like you wouldn't believe) but getting the tasks themselves might be a goer - hadn't been able to find them. Mostly we've been using msbuild scripts to get round such issues. Bit disingenuous to suggest that you can't have the same ci and cd use cases for on-premise as VS Online though (much as I like VSO and use it for my other client projects). – Steve Pettifer Mar 10 '16 at 07:13
  • Yeah we don't have just "Copy Files" either. There's Windows Machine Copy Files which requires setting up a server as a deploy target, and Azure Copy Files, which of course requires Azure. – Chaim Eliyah May 27 '16 at 19:56
  • 3
    Every night before I go to bed, I pull out a little statue of billg from my nightstand and pray a silent prayer that somebody will add a "flatten hierarchy" option to the Copy Files action. Such an option would restore a lot of sanity in my world. AFAICT I am now forced to resort to PowerShell for the file copying. CopyFiles looks unfinished as-is. – 9Rune5 Jun 07 '16 at 09:59
  • 1
    @9Rune5 It's finally there with the new Build Agents that come with TFS 2017 Update 1. Thank you for restoring my faith. – JamesQMurphy Sep 15 '17 at 20:50
7

"Flatten Folders" option in "Advanced" section of "Copy Files" step.

If you are using TFS Online (Visual Studio Online) and don't need to copy the folder structure use "Flatten Folders" option in "Advanced" section of "Copy Files" step in your build definition

igor_1024
  • 166
  • 1
  • 4
  • please provide details with your answer. – Sachith Muhandiram Feb 11 '17 at 01:25
  • If you are using TFS Online (Visual Studio Online) and don't need to copy the folder structure use "Flatten Folders" option in "Advanced" section of "Copy Files" step in your build definition. – igor_1024 Feb 12 '17 at 16:41
  • This is now available in the on-prem version as of TFS 2017, Update 1. The "Advanced" section was collapsed and I did not even see it at first. But lo and behold, "Flatten files" was there. – JamesQMurphy Sep 15 '17 at 20:46
  • Honestly, why is one of the simplest commands on any OS made so difficult? The official documentation is also useless with few, if any, examples. The flatten option doesn't explain what it is flattening and how. Microsoft really needs to get their act together. – ATL_DEV Mar 01 '21 at 23:18
4

With the new web based build system you can use multiple patterns in a single step. Therefore you can do something like this for your case:

Project\bin\x86\Release\project.exe
SomeScripts\**\*

Or if you have the build platform and configuration in a variable (eg. BuildPlatform / BuildConfiguration) which you also use in the build step you could use them in the pattern:

Project\bin\$(BuildPlatform)\$(BuildConfiguration)\project.exe
SomeScripts\**\*

If you want the project.exe to be in the root instead of the structure you need to use a Copy Task to stage your files in the desired structure first. You can use $(Build.StagingDirectory) as a target for this. Afterwards use the Publish task with $(Build.StagingDirectory) as copy root and publish everything from this root to the drop.

Pascal Berger
  • 4,262
  • 2
  • 30
  • 54
  • Not, it's not working as expected. if I make like that, I'll receive next directory structure: Project\bin\x86\Release\project.exe Project\obj\x86\Release\project.exe Project\bin\Release\project.exe Project\obj\Release\project.exe – Georgy Grigoryev Jan 04 '16 at 09:50
  • but I need only Project\bin\x86\Release\project.exe – Georgy Grigoryev Jan 04 '16 at 09:51
  • 2
    No, it's still copying with folder structure – Georgy Grigoryev Jan 04 '16 at 12:28
  • Updated the answer how to restructure files – Pascal Berger Jan 05 '16 at 07:15
  • Even if you use a $(Build.StagingDirectory) you can't use the patterns you described here. – Daniel Bişar Jan 22 '16 at 11:00
  • Why not? See the answer from Eddie with detailed screenshots how to set the parameters – Pascal Berger Jan 22 '16 at 11:04
  • The problem is the Source Folder. As Eddie described above you would have to set the Source Folder to the actual project or even binary folder. From there you execute the match. You said a minimatch like Project\bin\$(BuildPlatform)\$(BuildConfiguration)\project.exe and SomeScripts\\**\\* would create the desired result. But this is not the case. No matter if you use a staging dir or not. This is the same Georgy mentioned. – Daniel Bişar Jan 22 '16 at 11:08
  • Maybe may answer was not clear enough. The above minimatch patterns copy the structure. For changing the structure you need a copy task, with different minimatch patterns, as described by Eddie above. – Pascal Berger Jan 22 '16 at 11:16
4

The "flattenFolders" option is also available as a YAML task parameter. The following code excerpt shows a CopyFiles@2 task which copyies the build output to the $(Build.ArtifactStagingDirectory). When I specify the option flattenFolders: true the nested folder structure bin\release\...\My.exe is flattened, means the exe files is copied to the root of $(Build.ArtifactStagingDirectory).

- task: CopyFiles@2
  displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
  inputs:
    SourceFolder: '$(system.defaultworkingdirectory)'
    Contents: |
     **\bin\$(BuildConfiguration)\**\*.exe
    TargetFolder: '$(Build.ArtifactStagingDirectory)'
    flattenFolders: true

Further documentation concerning the CopyFiles task can be found here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/copy-files?view=vsts&tabs=yaml

thomasgalliker
  • 1,279
  • 13
  • 19
2

For those who would like to have a PowerShell script to use in your build server, here is a working (at least, on my build server ;)) sample:

param
(
    [string] $buildConfiguration = "Debug",
    [string] $outputFolder = $PSScriptRoot + "\[BuildOutput]\"
)

Write-Output "Copying all build output to folder '$outputFolder'..."

$includeWildcards = @("*.dll","*.exe","*.pdb","*.sql")
$excludeWildcards = @("*.vshost.*")

# create target folder if not existing, or, delete all files if existing
if(-not (Test-Path -LiteralPath $outputFolder)) {
    New-Item -ItemType Directory -Force -Path $outputFolder | Out-Null

    # exit if target folder (still) does not exist
    if(-not (Test-Path -LiteralPath $outputFolder)) {
        Write-Error "Output folder '$outputFolder' could not be created."
        Exit 1
    }
} else {
    Get-ChildItem -LiteralPath $outputFolder -Include * -Recurse -File | foreach {
        $_.Delete()
    }
    Get-ChildItem -LiteralPath $outputFolder -Include * -Recurse -Directory | foreach {
        $_.Delete()
    }
}

# find all output files (only when in their own project directory)
$files = @(Get-ChildItem ".\" -Include $includeWildcards -Recurse -File |
    Where-Object {(
        $_.DirectoryName -inotmatch '\\obj\\' -and
        $_.DirectoryName -inotmatch '\\*Test*\\' -and
        $_.DirectoryName -ilike "*\" + $_.BaseName + "\*" -and
        $_.DirectoryName -ilike "*\" + $buildConfiguration
    )}
)

# copy output files (overwrite if destination already exists)
foreach ($file in $files) {
    Write-Output ("Copying: " + $file.FullName)
    Copy-Item $file.FullName $outputFolder -Force

    # copy all dependencies from folder (also in subfolders) to output folder as well (if not existing already)
    $dependencies = Get-ChildItem $file.DirectoryName -Include $includeWildcards -Exclude $excludeWildcards -Recurse -File
    foreach ($dependency in $dependencies) {
        $dependencyRelativePathAndFilename = $dependency.FullName.Replace($file.DirectoryName, "")
        $destinationFileName = Join-Path -Path $outputFolder -ChildPath $dependencyRelativePathAndFilename
        if (-not(Test-Path -LiteralPath $destinationFileName)) {
            Write-Output ("Copying: " + $dependencyRelativePathAndFilename + " => " + $destinationFileName)

            # create sub directory if not exists
            $destinationDirectory = Split-Path $destinationFileName -Parent
            if (-not(Test-Path -LiteralPath $destinationDirectory)) {
                New-Item -Type Directory $destinationDirectory
            }
            Copy-Item $dependency.FullName $destinationDirectory
        } else {
            Write-Debug ("Ignoring (existing destination): " + $dependency.FullName)
        }
    }
}

Here is the script being used in a PowerShell build step:

TFS 2015 Build - Output to single folder step

DotBert
  • 1,262
  • 2
  • 16
  • 29
  • Do you know what does meant KISS?) I guess couple of steps is more easy than supporting that script – Georgy Grigoryev Jun 17 '16 at 09:48
  • I know @GeorgyGrigoryev - that is always something to consider. This one does one job and does it well, imho. Please feel free to chunk the file into anything you like, or even better, share your solution ;)... – DotBert Jun 30 '16 at 20:39
  • Yes, but I stopped at point that I should make my deploy script with folder structure and got rid from this problem. My solution is using one of the answers above. – Georgy Grigoryev Jul 02 '16 at 08:30
1

With TFS2017update1 and above, VSTS, you could simply check Flatten Folders under Advanced option in Copy Files Task. The easiest solution for now.

enter image description here

This will flatten the folder structure and copy all files into the specified target folder.

PatrickLu-MSFT
  • 49,478
  • 5
  • 35
  • 62
0

Make artifacts of each file you want to copy. Then create a 'copy file' task of each file of these artifacts. Then it doesn't copy the source tree structure.