Why is this PowerShell script so slow? How can I speed it up?

Question

I developed this script to apply Sitecore workflows to whole swaths of items without having to manually click through the GUI. I'm pretty pleased with how well it works, but it's just slow. Here is the script:

Import-Module 'C:\Subversion\CMS Build\branches\1.2\sitecorepowershell\Sitecore.psd1'

# Hardcoded IDs of workflows and states
$ContentApprovalWfId = "{7005647C-2DAC-4C32-8A09-318000556325}";
$ContentNoApprovalWfId = "{BCBE4080-496F-4DCB-8A3F-6682F303F3B4}";
$SettingsWfId = "{7D2BA7BE-6A0A-445D-AED7-686385145340}";

#new-psdrive *REDACTED*

set-location m-rocks:

function ApplyWorkflows([string]$path, [string]$WfId) {
    Write-Host "ApplyWorkflows called: " $path " - " $wfId;
    $items = Get-ChildItem -Path $path;
    $items | foreach-object {
        if($_ -and $_.Name) {
            $newPath = $path + '\' + $_.Name;
            $newPath;
        } else {
            Write-host "Name is empty.";
            return;
        }

        if($_.TemplateName -eq "Folder" -or $_TemplateName -eq "Template Folder") {
            # don't apply workflows to pure folders, just recurse
            Write-Host $_.Name " is a folder, recursing.";
            ApplyWorkflows $newPath $wfId;
        }
        elseif($_.TemplateName -eq "Siteroot" -or $_.TemplateName -eq "InboundSiteroot") {
            # Apply content-approval workflow
            Set-ItemProperty $newPath -name "__Workflow" $ContentApprovalWfId;
            Set-ItemProperty $newPath -name "__Default workflow" $ContentApprovalWfId;

            # Apply content-no-approval workflow to children

            Write-Host $_.Name " is a siteroot, applying approval workflow and recursing.";
            ApplyWorkflows $newPath $ContentNoApprovalWfId;
        }
        elseif($_.TemplateName -eq "QuotesHomePage") {
            # Apply settings workflow to item and children

            Write-Host $_.Name " is a quotes item, applying settings worfklow recursing.";

            Set-ItemProperty $newPath -name "__Workflow" $SettingsWfId;
            Set-ItemProperty $newPath -name "__Default workflow" $SettingsWfId;

            ApplyWorkflows $newPath $SettingsWfId;
        }
        elseif($_.TemplateName -eq "Wildcard")
        {
            Write-Host $_.Name " is a wildcard, applying workflow (and halting).";
            Set-ItemProperty $newPath -name "__Workflow" $ContentApprovalWfId;
            Set-ItemProperty $newPath -name "__Default workflow" $ContentApprovalWfId;
        }
        elseif($_ -and $_.Name) {
            # Apply passed in workflow and recurse with passed in workflow
            Write-Host $_.Name " is a something else, applying workflow and recursing.";
            Set-ItemProperty $newPath -name "__Workflow" $WfId;
            Set-ItemProperty $newPath -name "__Default workflow" $WfId;

            ApplyWorkflows $newPath $wfId;
        }
    }
}

ApplyWorkflows "sitecore\Content\" $ContentNoApprovalWfId;

It processes one item in a little less than a second. There are some pauses in its progress - evidence suggests that this is when Get-ChildItem returns a lot of items. There are a number of things I would like to try, but it's still running against one of our sites. It's been about 50 minutes and looks to be maybe 50% done, maybe less. It looks like it's working breadth-first, so it's hard to get a handle on exactly what's done and what's not.

So what's slowing me down?

Is it the path construction and retrieval? I tried to just get the children on the current item via $_ or $_.Name, but it always looks in the current working directory, which is the root, and can't find the item. Would changing the directory on every recursion be any faster?

Is it the output that's bogging it down? Without the output, I have no idea where it is or that it's still working. Is there some other way I could get indication of where it is, how many it has done, etc.?

Is there a better approach where I just use Get-ChildItem -r with filter sets and loop through those? If so, a first attempt at incorporating some of my conditionals in the first script into a filter set would be very much appreciated. I am new to PowerShell, so I'm sure there's more than one or two improvements to be made in my code.

Is it that I always call the recursive bit even if there aren't any children? The content tree here is very broad with a very many leaves with no children. What would be a good check whether or not child items exist?

Finally, the PowerShell provider (PSP) we have is not complete. It does not seem to have a working implementation of Get-Item, which is why everything is almost completely written with Get-ChildItem instead. Our Sitecore.Powershell.dll says it is version 0.1.0.0. Would upgrading that help? Is there a newer one?

Edit: it finally finished. I did a count on the output and came up with 1857 items and it took ~85 minutes to run, for an average of 21 items per minute. Slower than I thought, even...

Edit: My first run was on PowerShell 2.0, using the Windows PowerShell ISE. I've not tried the Sitecore PowerShell plugin module or the community. I didn't even know it existed until yesterday :-)

I tried another run after upgrading to PowerShell 3.0. Starting locally - running the script from my laptop, connecting to the remote server - there was no noticeable difference. I installed PowerShell 3.0 on the hosting box and ran the script from there and saw maybe a 20-30% increase in speed. So that's not the silver bullet I was hoping it would be - I need an order of magnitude or two's worth of improvement to make this something I won't have to babysit and run in batches. I am now playing around with some of the actual script improvements suggested by the fine answers below. I will post back with what works for me.

Answer 1

Personally I think the biggest boost you would get if you started using the community PowerShell implementation over the Rocks one.

Let me explain why.

You're traversing the whole tree which means you have to visit every node in your branch, which means it has to be read and travel over the Rocks web service at least once. Then every property save is another webservice call.

I have run your script in the community console and it took me around 25 seconds for 3724 items. (I've removed the modifications as the values didn't relate to my system).

A Simple

Get-ChildItem -recurse

On my 3724 item tree took 11 seconds in the community console vs 48 seconds in the Rocks implementation.

Additional tweaks you could use in the Community implementation for your script would be using the Sitecore query like:

get-item . -Query '/sitecore/content/wireframe//*[@@TemplateName="Template Folder"]'

and only send those items into your function

None of it means the Rocks console is not written right, it just means the design choices in the Rocks console and their target is different.

You can find the community console here: http://bit.ly/PsConScMplc

Answer 2

See this blog post where there are stated differences between foreach-object cmdlet and foreach statement.

You could speed it up when you pipe the get-childitem with foreach object:

get-childitem . | foreach-object { ApplyWorkflow($_) }

This will cause that each object returned by get-childitem will be immediately passed to the following step in pipeline so you will be processing them only once. This operation should also prevent long pauses for reading all children.

Additionally you can get all the items recursively and filter them by template and then apply appropriate workflows, eg:

get-childitem -recurse . | where-object { $_.TemplateName -eq "MyTemplate"} | foreach-object { ApplyWorkflowForMyTemplate($_) }
get-childitem -recurse . | where-object { $_.TemplateName -eq "MySecondTemplate"} | foreach-object { ApplyWorkflowForMySecondTemplate($_) }

Still I would rather not expect this script to run in seconds anyway. In the end you are going over the whole content tree.

And finally what library are you using? Is this Sitecore Powershell Console (the name of the dll sounds familiar)? There is newer version which has a lots of new features added.

Answer 3

The most obvious problems I see are that you are iterating over files twice per invocation of the function:

function ApplyWorkflows([string]$path, [string]$WfId) {
    Write-Host "ApplyWorkflows called: " $path " - " $wfId;


    # this iterates through all files and assigns to $items
    $items = Get-ChildItem -Path $path;

    # this iterates through $items for a second time
    $items | foreach-object {  # foreach-object is slower than for (...)

Also, piping to foreach-object is slower than using the traditional for keyword.