PowerShell Delete more quickly

Question

I have a PowerShell script that recursively deletes all files and folders, but excludes certain folders as they should not be deleted. It works 100%, but my problem is performance. I need this to run a lot faster.

Any ideas on how to make this faster?

Write-Host "Purging $InstallationDirectorySite - Deleting files..."

$FolderExlusions = (
    "App_Data",
    "Logs",
    "TEMP",
    "ExamineIndexes",
    "DistCache",
    "GitPathProviderRepository"
)

[regex] $files_regex = "Logs|ExamineIndexes|DistCache*|GitPathProviderRepository*"

if(Test-Path $InstallationDirectorySite) {
    Get-ChildItem -Path $InstallationDirectorySite -Recurse -Exclude $FolderExlusions |
        Where-Object {$_.FullName -notmatch $files_regex} |
        Remove-Item -Recurse
}
else {
    Write-Output "$InstallationDirectorySite doesn't exist"
}

You could compare the speed of com or .net methods instead of `rm -r`: https://social.technet.microsoft.com/Forums/office/en-US/14089f8a-65eb-4c31-8d87-2485963bda2b/quickly-delete-large-folders-with-ps?forum=winserverpowershell — marsze, Mar 11 '19 at 09:24
Consider using intermediate variables instead of piping. Combine that with `Measure-Command` to find out which part, exactly, is the slow one. How slow is the delete now and what kind of performance are you looking for? How many directories and files are you working with? — vonPryz, Mar 11 '19 at 09:24
The delete has to process 40 odd folders, but those folders contain a lot of files. The worst folder is the node modules folder that contains a ton of files. The delete at the moment is around 2 minutes, if I can halve that I would be happy. I will definitely look into Measure-Command for some testing — Michéle Johl, Mar 11 '19 at 09:43
NTFS is known to have its quirks with large number of files. See [earlier a question](https://stackoverflow.com/q/197162). TL;DR: disable 8.3 file names and last access time for 1000+ files / dir environments. — vonPryz, Mar 11 '19 at 10:44

score 0 · Accepted Answer · answered Mar 11 '19 at 10:37

You are in fact filtering the excluded folders twice.
The first time using the -Exclude parameter and the second time using a Regex -match.
However, the Exclude parameter takes a string array, not a single string with keywords separated by a comma and a newline as you get from the 'here-string'. See Get-ChildItem

Also, the regex you use is wrong, because the asteriks * in regex is a quantifier, not a wildcard character.

I suggest you filter once using either the -Exclude parameter like this (here the asteriks is a Wildcard):

$FolderExlusions = "App_Data","Logs","TEMP","ExamineIndexes","DistCache*","GitPathProviderRepository*"
Get-ChildItem -Path $InstallationDirectorySite -Recurse -Exclude $FolderExlusions | Remove-Item -Recurse -WhatIf

Or use only the regex method in a Where-Object clause like this:

$FolderExlusions = "^(App_Data|Logs|TEMP|ExamineIndexes|DistCache.*|GitPathProviderRepository.*)"
Get-ChildItem -Path $InstallationDirectorySite -Recurse | Where-Object { $_.Name -notmatch $FolderExlusions } | Remove-Item -Recurse -WhatIf

Remove the -WhatIf if you are satisfied with the results.

Hope that helps

I have been testing and the Where-Object {$_.FullName -notmatch $files_regex} actually works and stops the exclusion from deleting the folder and it's children. Without it the folder is not deleted, but the children are, which is not the result we want as we want to keep the folder and it's children. Does not look like I will get much gain either way. Time to move to SSD on servers then, — Michéle Johl, Mar 11 '19 at 13:03

PowerShell Delete more quickly

1 Answers1