$_ variable used in function from a module is empty (PowerShell)

Question

One question for you is here ;)

I have this function:

function Set-DbFile {
    param(
        [Parameter(ValueFromPipeline=$true)]
        [System.IO.FileInfo[]]
        $InputObject,
        [Parameter(ValueFromPipelineByPropertyName=$true)]
        [scriptblock]
        $Properties
    )
    process {
        $InputObject | % { 
            Write-Host `nInside. Storing $_.Name
            $props = & $Properties
            Write-Host '  properties for the file are: ' -nonew
            write-Host ($props.GetEnumerator()| %{"{0}-{1}" -f $_.key,$_.Value})
        }
    }
}

Look at the $Properties. It should be evaluated for each file and then the file and the properties should be processed further.

Example how to use it might be:

Get-ChildItem c:\windows |
    ? { !$_.PsIsContainer } |
    Set-DbFile -prop { 
        Write-Host Creating properties for $_.FullName
        @{Name=$_.Name } # any other properties based on the file
    }

When I copy & paste function Set-dbFile to command line and run the example snippet, everything is fine.

However, when I store the function in a module, import it and run the example, the $_ variable is empty. Does anybody know why? And how to solve it? (other solutions are welcome as well)

---

Results for function defined in a script/typed in commandline:

Inside. Storing adsvw.ini
Creating properties for C:\windows\adsvw.ini
  properties for the file are: Name-adsvw.ini

Inside. Storing ARJ.PIF
Creating properties for C:\windows\ARJ.PIF
  properties for the file are: Name-ARJ.PIF
....

Results for function defined in module:

Inside. Storing adsvw.ini
Creating properties for
  properties for the file are: Name-

Inside. Storing ARJ.PIF
Creating properties for
  properties for the file are: Name- 
....

Answer 1

The problem here is down to scope hierarchy. If you define two functions like...

function F1{
    $test="Hello"
    F2
}
function F2{
    $test
}

Then F2 will inherit the variable scope of F1 since it's called from F1's scope. If you define function F2 in a module and export the function the $test variable is not available since the module has it's own scope tree. See the Powershell Language Specification (Section 3.5.6):

In your case the current node variable is defined in the local scope and hence it will not survive into the module scope since it's in a different tree with a different scope root (apart from global variables).

To quote the text on the GetNewClosure() method in the Powershell Language Specification (Section 4.3.7):

Retrieves a script block that is bound to a module.Any local variables that are in the context of the caller will be copied into the module.

...hence GetNewClosure() works a treat since it bridges the local scope/module divide. I hope this helps.

Answer 2

Looks like GetNewClosure() is as good a work around as any, but it changes the way the script block sees those variables. Passing $_ to the scriptblock as an argument works, too.

It has nothing to do with normal scope issues (e.g., global vs local), but it appears like that at first. Here's my very simplified reproduction and some explanation following:

script.ps1 for normal dot-sourcing:

function test-script([scriptblock]$myscript){
    $message = "inside"
    &{write-host "`$message from $message"}    
    &$myscript
}

Module\MyTest\MyTest.psm1 for importing:

function test-module([scriptblock]$myscript){
    $message = "inside"
    &{write-host "`$message from $message"}    
    &$myscript
}

function test-module-with-closure([scriptblock]$myscript){
    $message = "inside"
    &{write-host "`$message from $message"}    
    &$myscript.getnewclosure()
}

Calls and output:

» . .\script.ps1

» import-module mytest

» $message = "outside"

» $block = {write-host "`$message from $message (inside?)"}

» test-script $block
$message from inside
$message from inside (inside?)

» test-module $block
$message from inside
$message from outside (inside?)

» test-module-with-closure $block
$message from inside
$message from inside (inside?)

So I started hunting around since this piqued my curiosity, and I found a few interesting things.

This Q&A, which also features a link to this bug report is pretty much the exact same topic, as are some other blog articles I ran across. But while it was reported as a bug, I disagree.

The about_Scopes page has this to say (w:

...

Restricting Without Scope

  A few Windows PowerShell concepts are similar to scope or interact with 
  scope. These concepts may be confused with scope or the behavior of scope.

  Sessions, modules, and nested prompts are self-contained environments,
  but they are not child scopes of the global scope in the session.

  ...

  Modules:
    ...

    The privacy of a module behaves like a scope, but adding a module
    to a session does not change the scope. And, the module does not have
    its own scope, although the scripts in the module, like all Windows
    PowerShell scripts, do have their own scope. 

Now I understand the behavior, but it was the above and a few more experiments that led me to it:

• If we change $message in the scriptblock to $local:message then all 3 tests have a blank space, because $message is not defined in the scriptblock's local scope.
• If we use $global:message, all 3 tests print outside.
• If we use $script:message, the first 2 tests print outside and the last prints inside.

Then I also read this in about_Scopes:

Numbered Scopes:
    You can refer to scopes by name or by a number that
    describes the relative position of one scope to another.
    Scope 0 represents the current, or local, scope. Scope 1
    indicates the immediate parent scope. Scope 2 indicates the
    parent of the parent scope, and so on. Numbered scopes
    are useful if you have created many recursive
    scopes.

• If we use $((get-variable -name message -scope 1).value) in order to attempt getting the value from the immediate parent scope, what happens? We still get outside rather than inside.

At this point it was clear enough to me that sessions and modules have their own declaration scope or context of sorts, at least for script blocks. The script blocks act like anonymous functions in the environment in which they're declared until you call GetNewClosure() on them, at which point they internalize copies of the variables they reference of the same name in the scope where GetNewClosure() was called (using locals first, up to globals). A quick demonstration:

$message = 'first message'
$sb = {write-host $message}
&$sb
#output: first message
$message = 'second message'
&$sb
#output: second message
$sb = $sb.getnewclosure()
$message = 'third message'
&$sb
#output: second message

I hope this helps.

Addendum: Regarding design.

JasonMArcher's comment made me think about a design issue with the scriptblock being passed into the module. In the code of your question, even if you use the GetNewClosure() workaround, you have to know the name of the variable(s) where the scriptblock will be executed in order for it to work.

On the other hand, if you used parameters to the scriptblock and passed $_ to it as an argument, the scriptblock does not need to know the variable name, it only needs to know that an argument of a particular type will be passed. So your module would use $props = & $Properties $_ instead of $props = & $Properties.GetNewClosure(), and your scriptblock would look more like this:

{ (param [System.IO.FileInfo]$fileinfo)
    Write-Host Creating properties for $fileinfo.FullName
    @{Name=$fileinfo.Name } # any other properties based on the file
}

See CosmosKey's answer for further clarification.

Answer 3

I believe you need to call getnewclosure() on that script block before you run it. Called from a script file or module, script blocks are evaluated at compile time. When you work from the console, there is no "compile time". It's evaluated at run time, so it behaves differenly there than when it's in the module.