Powershell: The specified path, or filename, or both, are too long - Directory Excludes

Question

We’ve been running this script fine previously, but recently we’ve been getting some issues (See below error) with our config transform step from our TFS Build.

$serviceFiles = Get-ChildItem $localWorkspace -Recurse | where {$_.Extension -eq ".exe"}

We recently switched to using Gulp to compile our CSS and JS which has given us a “node_modules” folder. It looks like it’s trying to crawl these folders and actually gets to the directory length limit. I’ve tried various suggestions I’ve found from googling and other related questions on SO, but none seem to be working for me, it’s still hitting the same error (and I assume isn’t actually excluding these folders)

This is an example of a modified version I've tried using to exclude the folders

$excludedDirectories = @("*node_modules*", "*packages*", "*Common*");
Write-Verbose $localWorkspace

# get services for config types (service configs named same as assmebly .exe)
$serviceFiles = Get-ChildItem $localWorkspace -Recurse -Exclude $excludedDirectories | ?{$_.Extension -eq ".exe" -and $_.FullName -notmatch '*node_modules*' }

I've tried some variations on this based on other SO questions and answers, but the solution evades me. I've read from a few sources that the -Exclude doesn't work for a lot of people in most situations, so i tried the solution of a where clause to exclude a folder (I want to exclude multiple, but I tried just node_modules to see if I could get it past that, the other folders aren't too deep)

I want to exclude the node_modules directory, along with a couple of others that don't need to be checked for a transform. Any help would be greatly appreciated, thank you.

**Do *not* use images for code. Post the actual code using the standard formatting tools.** (-1 until you fix this.) RE your issue: I don't have anything definite to offer, but my first thought when I saw the title was, "node_modules," and low an behold, there it is in your excludes. node_modules has a tendency to create a nested hierarchy that's so deep it starts running into Windows file length limitations. — jpmc26, Jul 25 '16 at 20:43
Actually, something for you. I'm having trouble finding documentation of it, but as I recall, `-Exclude`s are processed *after* the underlying system obtains all the files. So `-Exclude` wouldn't work. — jpmc26, Jul 25 '16 at 20:47
Thank you, I've amended my original post with code snippets rather than images. — ICTech, Jul 25 '16 at 20:57
Duplicate of someone else asking to exclude *node_modules* folder four hours before you... http://stackoverflow.com/questions/38571839/exclude-directory-from-ls-recursive-search ? — TessellatingHeckler, Jul 25 '16 at 21:21
Images should only be used when necessary to demonstrate an issue that cannot be shown another way. Your error messages and code are both text, and can be copied/pasted directly into your question as such. Please see [this Meta post](http://meta.stackoverflow.com/a/285557/62576) for a list of the many reasons to avoid using images in your posts. — Ken White, Jul 25 '16 at 23:27
@KenWhite - I'm aware of that seeing as jpmc26 mentioned this before you commented. My error message is an image but I thought that would be alright. — ICTech, Jul 26 '16 at 07:23
Read the Meta post I linked before. As I said, it lists many reasons to **avoid using images in your post**, whether it's code or an error message or a picture of your favorite pet. — Ken White, Jul 26 '16 at 12:30

jpmc26 · Accepted Answer · 2016-07-26T01:44:09.850

Unfortunately, the -Exclude parameter won't exclude the directories before enumerating all the files. So it still errors out. You need to exclude them earlier than that.

If those directories only exist at the top level, you can enumerate the top level directories, exclude the ones you don't want, and then examine the contents inside the remaining directories. Like this:

$excludedDirectories = @("node_modules", "packages", "Common")

$serviceFiles = Get-ChildItem $localWorkspace -Exclude $excludedDirectories | % { Get-ChildItem $_ -Recurse } | ? {$_.Extension -eq ".exe" }

Also, you can make this a bit simpler by using -Include:

$serviceFiles = Get-ChildItem $localWorkspace -Exclude $excludedDirectories | % { Get-ChildItem $_ -Recurse -Include "*.exe" }

Notice what I did. I removed the top level -Recurse and filtered out those directories. Then I used -Recurse on the remaining children of the top-most parent, giving us the files we're looking for.

If the directories you need to filter out appear deep in the hierarchy or at multiple levels, you'll have to write your own recursive traversal function:

function Get-ChildItemRecursiveExclude(
    [Parameter(Mandatory=$true)][string[]]$Path,
    [Parameter(Mandatory=$true)][string[]]$ExcludedDirNames
) {
    $immediateChildren = Get-ChildItem $Path -Exclude $ExcludedDirNames
    foreach ($c in $immediateChildren) {
        # Uncaptured output is returned
        $c
        if (Test-Path $c -PathType Container) {
            Get-ChildItemRecursiveExclude $c $ExcludedDirNames
        }
    }
}

$serviceFiles = Get-ChildItemRecursiveExclude $localWorkspace @("node_modules", "packages", "Common") | ? { $_.Extension -eq ".exe" }

Overall, the basic idea is that you have to keep PowerShell from traversing down into node_modules in the first place. npm creates very deep hierarchies that surpass the old limits on path lengths. (I'm not really clear why .NET still enforces them, but it does even though some of the underlying Windows API no longer does. for example, robocopy and several third party runtimes, like Node, don't bother with them.)

Powershell: The specified path, or filename, or both, are too long - Directory Excludes

1 Answers1