12

I'm setting up TFS 2015 on-prem and I'm having an issue on my last build step, Publish Build Artifacts. For some reason, the build agent appears to be archiving old binaries and I'm left with a huge filepath: 

E:\TFSBuildAgent\_work\1a4e9e55\workspace\application\Development\project\WCF\WCF\obj\Debug\Package\Archive\Content\E_C\TFSBuildAgent\_work\1a4e9e55\workspace\application\Development\project\WCF\WCF\obj\Debug\Package\PackageTmp\bin

I'm copying the files using the example minimatch pattern to begin with: 

**\bin

I'm only testing at the moment so this is not a permanent solution but how can I copy all binaries that are in a bin folder but not a descendant of obj?

From research I think that this should work, but it doesn't (It doesn't match anything):

**!(obj)**\bin

I'm using www.globtester.com to test. Any suggestions?

On a separate note, I'll look into the archiving issue later but if anyone has any pointers on it, feel free to comment. Thanks

Rodders
  • 2,425
  • 2
  • 20
  • 34
  • As to the archiving issue, I suspect that your definition is not configured to clean the workspace or call `/t:rebuild` or `/t:clean,build` on the msbuild commandline, which will trigger an incremental build, which is faster, but can leave old bits around. – jessehouwing Aug 25 '15 at 12:38
  • Also look into copying everything you want to drop to the `$(Build.ArtifactsStagingDirecotry)`, that folder will be cleared automatically after each build. See: https://msdn.microsoft.com/Library/vs/alm/Build/scripts/variables – jessehouwing Aug 25 '15 at 12:40

2 Answers2

13

In VSTS there are two kinds of pattern matching for URLs that are built-in to the SDKs. Most tasks nowadays use the Minimatch pattern as described in Matt's answer. However, some use the pattern that was used by the 1.x Agent's Powershell SDK. That format is still available in the 2.x Agent's Powershell SDK by the way.

So that means there are 5 kinds of tasks:

  • 1.x agent - Powershell SDK
  • 2.x agent - Node SDK
  • 2.x agent - Powershell 1 Backwards compatibility
  • 2.x agent - Powershell 3 SDK - Using find-files
  • 2.x agent - Powershell 3 SDK - Using find-match

The ones in bold don't Minimatch, but the format documented in the VSTS-Task-SDK's find-files method.

The original question was posted in 2015, at which point in time the 2.x agent wasn't yet around. In that case, the pattern would, in all likelihood, be:

 **\bin\$(BuildConfiguration)\**\*;-:**\obj\**

The -: excludes the items from the ones in front of it.

jessehouwing
  • 106,458
  • 22
  • 256
  • 341
  • 1
    Thanks for the reply. I can't get this to work though. It looks logical but does not select anything. Your other suggestion to do a rebuild instead works the first time but then fails after. It's making me think I have something fundamentally wrong. – Rodders Aug 26 '15 at 14:14
  • 1
    Sounds fishy... Did you use the build configuration variable as part of your msbuild call? Could you post screenshots? – jessehouwing Aug 26 '15 at 14:36
  • I've seen this syntax in the Visual Studio Test build step. But in the copy files section (Contents) it didn't work for me. For example, appending it like `**\?(*.exe|*.dll|*.pdb|*.xml|*.resx);-:**\obj\**` wasn't excluding obj ... – Matt Jul 19 '17 at 11:21
  • Can you share a reference for `-:`? In the [doc](https://learn.microsoft.com/en-us/vsts/pipelines/tasks/file-matching-patterns?view=vsts#exclude-patterns), it is stated that `!` is used for exclude pattern.. – yakya Aug 23 '18 at 08:12
  • 1
    Any of the tasks that rely on `find-vstsfiles` or are powershell tasks that use the V1 SDK that shipped with 2015. https://github.com/Microsoft/vsts-task-lib/blob/master/powershell/Docs/FullHelp/Find-VstsFiles.md – jessehouwing Aug 23 '18 at 09:32
  • @jessehouwing Cool, why you could always find some docs which hidden deep. Any specific trick to share? – Chamberlain Oct 21 '19 at 09:55
  • 1
    @chamberlain I've built a lot of tasks and have been using these repos por a long while. Knowing what to look for helps as well. I try to figure out how Azure Pipelines Tasks does things. – jessehouwing Oct 21 '19 at 12:24
  • Issues with this answer: (1) "So that means there are 4 kinds of tasks" but 5 are listed; (2) I can find no reference to `find-files` or `find-match` anywhere, only `Find-VsTsFiles`. Were the two former renamed and/or consolidated into the latter? – Ian Kemp Jan 24 '20 at 12:22
  • When the PowerShell module is imported it defines a prefix for methods. So find-files becomes find-vstsfiles. But the method in the psm1 file is defined as find-files. – jessehouwing Jan 24 '20 at 21:07
4

According to Microsoft's documentation, here is a list of file matching patterns you can use. The most important rules are:

Match with ?

  • ? matches any single character within a file or directory name (zero or one times).

Match with * or +

  • * or + matches zero or more characters within a file or directory name.

Match with @ sign

  • @ matches exactly once.

Match with Brackets (, ) and |

  • If you're using brackets with | it is treated as a logical OR, e.g. *(hello|world) means "Zero or more occurrances of hello or world"

Match with Double-asterisk **

  • ** recursive wildcard. For example, /hello/**/* matches all descendants of /hello.

Exclude patterns with !

  • Leading ! changes the meaning of an include pattern to exclude. Interleaved exclude patterns are supported.

Character sets with [ and ]

  • [] matches a set or range of characters within a file or directory name.

Comments with #

  • Patterns that begin with # are treated as comments.

Escaping

  • Wrapping special characters in [] can be used to escape literal glob characters in a file name. For example the literal file name hello[a-z] can be escaped as hello[[]a-z].

Example

The following expressions can be used in the Contents field of the "Copy Files" build step to create a deployment package for a web project:

**\?(.config|.dll|*.sitemap)
**\?(.exe|.dll|.pdb|.xml|*.resx)
**\?(.js|.css|.html|.aspx|.ascx|.asax|.Master|.cshtml|*.map)
**\?(.gif|.png|.jpg|.ico|*.pdf)

Note: You might need to add more extensions, depending on the needs of your project.

Ian Kemp
  • 28,293
  • 19
  • 112
  • 138
Matt
  • 25,467
  • 18
  • 120
  • 187
  • This is partially true. Unfortunately the PowerShell and TypeScript versions of the tasks use different minimatch libraries and what works in one task may not work in another. This is especially true for users on TFS 2015 and 2017, which have older task implementations. – jessehouwing Aug 18 '18 at 17:36
  • @jessehouwing - I was wondering why MS has chosen minimatch and not commonly known regular expressions - or just wildcards as we know them from the "good old" dir command. – Matt Aug 18 '18 at 19:14
  • It's the same format as for gitignore and a whole bunch of open source conventions. With the ms♥️OSS thing, I have an idea what triggered that move... – jessehouwing Aug 18 '18 at 21:12
  • @jessehouwing - exactly ;-) – Matt Aug 19 '18 at 07:34