Powershell | Speed up Running Get-ADUser in Powershell jobs

Question

I'm working on a tool that can quickly run through a csv and search AD for the relevant information provided I have made a gui that will let the user align the header of a CSV with the AD search method

My main issue at the moment is when searching AD for EmailAddress variable i am often getting the error "A connection to the directory on which to process the request was unavailable. This is likely a transient condition."

and its erroring out on select entries Limiting the number of powershell jobs running at any one time appears to assist with the issue but does not eliminate it entirely

Here is what I'm working with at the moment

$maxConcurrentJobs=15
$CheckBlock = {
            param ($User)
            Try { Get-ADUser -Filter { EmailAddress -eq $User } -Properties SamAccountName, EmployeeID, EmailAddress }
            Catch { Return "$User - $_" }
        }
        $Array.($listbox_Columns.SelectedItem) | ForEach-Object{
            $Check = $false 
            while ($Check -eq $false)
            {
                if ((Get-Job -State 'Running').Count -lt $maxConcurrentJobs)
                {
                    Write-Host "Processing EmailAddress $_"
                    Start-Job -ScriptBlock $CheckBlock -ArgumentList $_
                    $Check = $true
                }
            }
        }

Answer 1

I'm sorry this won't fit in a comment, but I'd been watching this discussion with the intent of chiming in.

While I'd be hard-pressed to cite references it's well known to me that the -Filter argument is interpreted/translated into an LDAP query by the AD cmdlets, on the client-side and then forwarded to the server as an LDAP query string.

This can be proven by upping the NTDS logging on a DC executing a -Filter query to see what's logged, my bet is it'll be the translated LDAP query. Moreover, it stands to reason because AFAIK AD LDAP (un-interfaced) can't answer any other type of query. My intent to double-check using this approach is the reason for my delayed participation.

Over the years, I've repeatedly tested performance between -Filter & -LDAPFilter and repeatedly came up with extremely narrow differences swinging in either direction. Given general performance variability, my conclusion is there's little to no difference! While we can assume there's some overhead involved in the interpretation of the -Filter argument it's probably minimal to the point of being undetectable. Included in that overhead there's an ability to query on calculated properties like "Enabled". That property as returned by Get-ADUser is likely a bitwise interpretation of UserAccountControl, it nevertheless can be queried with -Filter. There's been some debate about that and other properties/attributes, but I can personally attest to its reliability and have mentioned it in other SO discussions.

Note: It's possible these results could be different with more complex queries. However, increasing complexity might lead one to use -LDAPQuery for other more straightforward reasons.

I can't find those discussions at the moment but will try to update this note when I do. I know I reached a similar conclusion and commented on one of @mklement0's answers to which he directed me to one of @tomolak's where I logged a similar comment.

I've also long recognized that broad singular queries for broad needs are much faster than re-running a Get-* many times. Within reason, this appears to be universal, not specific to AD cmdlets. If I need to check that a user exists thousands of times it's much faster to load a list of all users first, then check the list than it is to run Get-ADUser that many times. Of course, if I only need to check a few users the formula might swing in the other direction. This firm observation is likely the reason the code sped up, not any difference between -Filter and -LDAPFilter.

In my experience, the real use case for -LDAPFilter is when certain attributes are not queryable with -Filter. This is likely due to a lack of Filter > LDAPFilter translation for the given property. So, the best advice I can give and that which I employ is to use -Filter until you can't, and switch to -LDAPFilter when you need. I can't rule out other use cases, perhaps portability of query strings between different AD cmdlets? Despite all this, if you're comfortable with LDAP Queries generally or specifically, there's certainly no functional harm in using them.

Answer 2

I'd suggest moving to -LDAPFilter, like this:

Get-ADUser -LDAPFilter "(mail=$User)" -Properties SamAccountName, EmployeeID, EmailAddress

And building from that, the optimum would be a single search that gets all users in one go. This can also be done with -LDAPFilter, but it requires a bit more doing.

$mails = $listbox_Columns.SelectedItem  # this should be an array of email addresses
$filter = $mails -join ')(mail='

Get-ADUser -LDAPFilter "(|(mail=$filter))" -Properties SamAccountName, EmployeeID, EmailAddress

Chances are high that you don't need to distribute this across many jobs anymore at this point, this is as efficient as it gets.

What's happening in the second code sample:

$mails -join ')(mail=' together with (|(mail=$filter)) creates an LDAP search expression in the form of (|(mail=A)(mail=B)(mail=C)) and so on, which would give you all the matching objects (but nothing else) in one server round-trip.

Of course you need to familiarize yourself with the LDAP search filter syntax and take a look at the raw LDAP property values in your AD to use it effectively, but this is a small price to pay for the performance gains it provides.