Increasing stack size with WinRM for ScriptMethods

Question

We are currently refactoring our administration scripts. It had just appeared that a combination of WinRM, error handling and ScriptMethod dramatically decreases available recursion depth.

See the following example:

Invoke-Command -ComputerName . -ScriptBlock {
    $object = New-Object psobject
    $object | Add-Member ScriptMethod foo {
        param($depth)
        if ($depth -eq 0) {
            throw "error"
        }
        else {
            $this.foo($depth - 1)
        }
    }

    try {
        $object.foo(5) # Works fine, the error gets caught
    } catch {
        Write-Host $_.Exception
    }

    try {
        $object.foo(6) # Failure due to call stack overflow
    } catch {
        Write-Host $_.Exception
    }
}

Just six nested calls are enough to overflow the call stack! Indeed, more than 200 local nested calls work fine, and without the try-catch the available depth doubles. Regular functions are also not that limited in recursion.

Note: I used recursion only to reproduce the problem, the real code contains many different functions on different objects in different modules. So trivial optimizations as "use functions not ScriptMethod" require architectural changes

Is there a way to increase the available stack size? (I have an administrative account.)

If I use something like `throw "MY ERROR"` then I cannot repro the problem (at least not as it is described). — Roman Kuzmin, Jan 24 '17 at 11:33
@RomanKuzmin this code can be executed as is (if WinRM on local machine enabled) — Pavel Mayorov, Jan 24 '17 at 11:46
@RomanKuzmin i fixed this syntax error ... and recursion limit has increased by 1. This is not enough :) — Pavel Mayorov, Jan 24 '17 at 11:57
If I use `. {...}` instead of `Invoke-Command` then the depth 134 is fine (135 overflows). I think it is not about just `ScriptMethod`, it is about the combination of it and `Invoke-Command` which *presumably* adds own nested calls. — Roman Kuzmin, Jan 24 '17 at 12:26
@RomanKuzmin *"It had just appeared that a combination of WinRM, error handling and ScriptMethod"* — Pavel Mayorov, Jan 24 '17 at 12:55
Would you mind sharing the real code? I mean, it's all fun and games trying to fix a problem in theory. But there might be an actual solution for your specific problem. — Akaino, Jan 25 '17 at 13:41
@Akaino the alternative solution is rewrite all deploy scripts using different architectural patterns. We are working on it now. But here i ask about stack size. — Pavel Mayorov, Jan 25 '17 at 13:45

score 1 · Answer 1 · answered Jan 25 '17 at 13:08

You have two problems that conspire to make this difficult. Neither is most effectively solved by increasing your stack size, if such a thing is possible (I don't know if it is).

First, as you've experienced, remoting adds overhead to calls that reduces the available stack. I don't know why, but it's easily demonstrated that it does. This could be due to the way runspaces are configured, or how the interpreter is invoked, or due to increased bookkeeping -- I don't know the ultimate cause(s).

Second and far more damningly, your method produces a bunch of nested exceptions, rather than just one. This happens because the script method is, in effect, a script block wrapped in another exception handler that rethrows the exception as a MethodInvocationException. As a result, when you call foo(N), a block of nested exception handlers is set up (paraphrased, it's not actually PowerShell code that does this):

try {
    try {
         ...
         try {
             throw "error"
         } catch {
             throw [System.Management.Automation.MethodInvocationException]::new(
                 "Exception calling ""foo"" with ""1"" argument(s): ""$($_.Exception.Message)""", 
                 $_.Exception
             )
         }
         ...
     } catch {
         throw [System.Management.Automation.MethodInvocationException]::new(
             "Exception calling ""foo"" with ""1"" argument(s): ""$($_.Exception.Message)""", 
             $_.Exception
         )
     }
 } catch {
     throw [System.Management.Automation.MethodInvocationException]::new(
         "Exception calling ""foo"" with ""1"" argument(s): ""$($_.Exception.Message)""", 
         $_.Exception
     )
 }

This produces a massive stack trace that eventually overflows all reasonable boundaries. When you use remoting, the problem is exacerbated by the fact that even if the script executes and produces this huge exception, it (and any results the function does produce) can't be successfully remoted -- on my machine, using PowerShell 5, I don't get a stack overflow error but a remoting error when I call foo(10).

The solution here is to avoid this particular deadly combination of recursive script methods and exceptions. Assuming you don't want to get rid of either recursion or exceptions, this is most easily done by wrapping a regular function:

$object = New-Object PSObject
$object | Add-Member ScriptMethod foo {
    param($depth)

    function foo($depth) {
        if ($depth -eq 0) {
            throw "error"
        }
        else {
            foo ($depth - 1)
        }
    }
    foo $depth
}

While this produces much more agreeable exceptions, even this can quite quickly run out of stack when you're remoting. On my machine, this works up to foo(200); beyond that I get a call depth overflow. Locally, the limit is far higher, though PowerShell gets unreasonably slow with large arguments.

As a scripting language, PowerShell wasn't exactly designed to handle recursion efficiently. Should you need more than foo(200), my recommendation is to bite the bullet and rewrite the function so it's not recursive. Classes like Stack<T> can help here:

$object = New-Object PSObject
$object | Add-Member ScriptMethod foo {
    param($depth)

    $stack = New-Object System.Collections.Generic.Stack[int]
    $stack.Push($depth)

    while ($stack.Count -gt 0) {
        $item = $stack.Pop()
        if ($item -eq 0) {
            throw "error"
        } else {
            $stack.Push($item - 1)
        }
    }
}

Obviously foo is trivially tail recursive and this is overkill, but it illustrates the idea. Iterations could push more than one item on the stack.

This not only eliminates any problems with limited stack depth but is a lot faster as well.

I used recursion only for demo - in real code there are 5 different functions on different objects. — Pavel Mayorov, Jan 25 '17 at 13:26
@PavelMayorov: you mean, mutual recursion between different script methods? In that case, yes, my approach won't help you. In that case, I put it to you that you are Boned (tm) if a rewrite isn't an option, unless someone does come up with some clever way to tweak the stack size. Note that the problem with nested exception handlers remains an actual, functional problem for such script methods. It sounds like your code base is using script methods as a way to do O-O in PowerShell, which is a nice idea in theory, but not in practice. — Jeroen Mostert, Jan 25 '17 at 13:29
@PavelMayorov: what about exception handling? If you have neither recursion nor an exception in a deeply nested method, you should be able to take the calls pretty far. If the problem is a deeply nested exception, that's another matter. — Jeroen Mostert, Jan 25 '17 at 13:32
Real exception is a FileNotFoundException. This exception cannot be avoided (attempts to test file existence lead to race conditions). — Pavel Mayorov, Jan 25 '17 at 13:37
Anyway, unexpected exceptions always happens. Root exception handling is the only way to locate bugs that happen only on Friday the thirteenth, and the full moon exactly at 13:13 and when the boss is looking. — Pavel Mayorov, Jan 25 '17 at 13:43
@PavelMayorov: yes, I'm not suggesting you don't use exceptions; I was just verifying what your actual problem case is, since it's oversimplified in your original question. In that case, my answer simply doesn't apply, nor can I think of any solution beyond petitioning the [team](https://github.com/powershell) to implement script methods differently (since that really is an issue in its current form). — Jeroen Mostert, Jan 25 '17 at 13:49

score 0 · Answer 2 · answered Jul 12 '17 at 13:44

0

Might be worth checking this out if you are overrunning the available memory in your remote session: Running Java remotely using PowerShell

I know it's for running a Java app but the solution updates the max memory available to a remote WinRM session.

answered Jul 12 '17 at 13:44

Ty Savercool

1,132
5
10

I have enough memory for this simple script (1Gb). Root of the problem is the stack size. – Pavel Mayorov Jul 12 '17 at 13:58

Increasing stack size with WinRM for ScriptMethods

2 Answers2