17

I have a Windows batch file whose purpose is to set some environment variables, e.g.

=== MyFile.cmd ===
SET MyEnvVariable=MyValue

Users can run this prior to doing work that needs the environment variable, e.g.:

C:\> MyFile.cmd
C:\> echo "%MyEnvVariable%"    <-- outputs "MyValue"
C:\> ... do work that needs the environment variable

This is roughly equivalent to the "Developer command prompt" shortcuts installed by Visual Studio, which set environment variables needed to run VS utilities.

However if a user happens to have a Powershell prompt open, the environment variable is of course not propagated back to Powershell:

PS C:\> MyFile.cmd
PS C:\> Write-Output "${env:MyEnvVariable}"  # Outputs an empty string

This can be confusing for users who switch between CMD and PowerShell.

Is there a way I can detect in my batch file MyFile.cmd that it was called from PowerShell, so that I can, for example, display a warning to the user? This needs to be done without any 3rd party utility.

mklement0
  • 382,024
  • 64
  • 607
  • 775
Joe
  • 122,218
  • 32
  • 205
  • 338
  • 2
    [This](https://stackoverflow.com/a/7486816/52598) might give you something to work with but note that the link between a parent and child process is entirely superficial. The kernel in no way guarantees that a PID doesn't get reused should your parent process die *(likely not an issue for your use case)*. – Lieven Keersmaekers Nov 23 '18 at 13:14
  • @LievenKeersmaekers - looks like a good start: I'd need to find the current process id then check if an ancestor name contains the string "powershell". – Joe Nov 23 '18 at 13:18
  • 1
    Well, [this](https://serverfault.com/a/126643/3679) might give you something to work with ;) – Lieven Keersmaekers Nov 23 '18 at 13:20
  • 1
    You could also use the [`setx` command](http://ss64.com/nt/setx.html) to permanently set environment variables, so they were also available in PowerShell later... – aschipfl Nov 23 '18 at 13:46
  • @LievenKeersmaekers - thanks again. It's too slow for my purposes, really needs to be instantaneous, but a promising start. Also I prefer not to change title because my batch file also sets a title and console color as feedback to the user. – Joe Nov 23 '18 at 13:58
  • 1
    @aschipfl thanks, but I specifically don't want them to be permanent. – Joe Nov 23 '18 at 14:01

4 Answers4

11

Your own answer is robust and while it is generally slow due to needing to run a PowerShell process, it can be made significantly faster by optimizing the PowerShell command used to determine the calling shell:

@echo off
setlocal
CALL :GETPARENT PARENT
IF /I "%PARENT%" == "powershell" GOTO :ISPOWERSHELL
IF /I "%PARENT%" == "pwsh" GOTO :ISPOWERSHELL
endlocal

echo Not running from Powershell 
SET MyEnvVariable=MyValue

GOTO :EOF

:GETPARENT
SET "PSCMD=$ppid=$pid;while($i++ -lt 3 -and ($ppid=(Get-CimInstance Win32_Process -Filter ('ProcessID='+$ppid)).ParentProcessId)) {}; (Get-Process -EA Ignore -ID $ppid).Name"

for /f "tokens=*" %%i in ('powershell -noprofile -command "%PSCMD%"') do SET %1=%%i

GOTO :EOF

:ISPOWERSHELL
echo. >&2
echo ERROR: This batch file must not be run from a PowerShell prompt >&2
echo. >&2
exit /b 1

On my machine, this runs about 3 - 4 times faster (YMMV) - but still takes almost 1 second.

Note that I've added a check for process name pwsh as well, so as to make the solution work with PowerShell Core too.


Much faster alternative - though less robust:

The solution below relies on the following assumption, which is true in a default installation:

Only a system environment variable named PSModulePath is persistently defined in the registry (not also a user-specific one).

The solution relies on detecting the presence of a user-specific path in PSModulePath, which PowerShell automatically adds when it starts.

@echo off
echo %PSModulePath% | findstr %USERPROFILE% >NUL
IF %ERRORLEVEL% EQU 0 goto :ISPOWERSHELL

echo Not running from Powershell 
SET MyEnvVariable=MyValue

GOTO :EOF

:ISPOWERSHELL
echo. >&2
echo ERROR: This batch file must not be run from a PowerShell prompt >&2
echo. >&2
exit /b 1

Alternative approach for launching a new cmd.exe console window on demand:

Building on the previous approach, the following variant simply re-invokes the batch file in a new cmd.exe window on detecting that it is being run from PowerShell.

This is not only more convenient for the user, it also mitigates the problem of the solutions above yielding false positives: When run from an interactive cmd.exe session that was launched from PowerShell, the above solutions will refuse to run, even though they should, as PetSerAl points out.
While the solution below also doesn't detect this case per se, it still opens a useable - albeit new - window with the environment variables set.

@echo off
REM # Unless already being reinvoked via cmd.exe, see if the batch
REM # file is being run from PowerShell.
IF NOT %1.==_isNew. echo %PSModulePath% | findstr %USERPROFILE% >NUL
REM # If so, RE-INVOKE this batch file in a NEW cmd.exe console WINDOW.
IF NOT %1.==_isNew. IF %ERRORLEVEL% EQU 0 start "With Environment" "%~f0" _isNew & goto :EOF

echo Running from cmd.exe, setting environment variables...

REM # Set environment variables.
SET MyEnvVariable=MyValue

REM # If the batch file had to be reinvoked because it was run from PowerShell,
REM # but you want the user to retain the PowerShell experience,
REM # restart PowerShell now, after definining the env. variables.
IF %1.==_isNew. powershell.exe

GOTO :EOF

After setting all environment variables, note how the last IF statement, also re-invokes PowerShell, but in the same new window, based on the assumption that the calling user prefers working in PowerShell.
The new PowerShell session will then see newly defined environment variables, though note that you'll need two successive exit calls to close the window.

mklement0
  • 382,024
  • 64
  • 607
  • 775
  • 1
    Thanks for this. I didn't get the 3-4 times performance improvement with your first version that you did, but your second version is, as you say, much faster, and plenty robust enough for my use case. – Joe Nov 25 '18 at 09:37
3

As Joe Cocker used to say "I get by with a little help from my friends".

In this case from Lieven Keersmaekers, whose comments led me to the following solution:

@echo off
setlocal
CALL :GETPARENT PARENT
IF /I "%PARENT%" == "powershell.exe" GOTO :ISPOWERSHELL
endlocal

echo Not running from Powershell 
SET MyEnvVariable=MyValue

GOTO :EOF

:GETPARENT
SET CMD=$processes = gwmi win32_process; $me = $processes ^| where {$_.ProcessId -eq $pid}; $parent = $processes ^| where {$_.ProcessId -eq $me.ParentProcessId} ; $grandParent = $processes ^| where {$_.ProcessId -eq $parent.ParentProcessId}; $greatGrandParent = $processes ^| where {$_.ProcessId -eq $grandParent.ParentProcessId}; Write-Output $greatGrandParent.Name

for /f "tokens=*" %%i in ('powershell -command "%CMD%"') do SET %1=%%i

GOTO :EOF

:ISPOWERSHELL
echo.
echo ERROR: This batch file may not be run from a PowerShell prompt
echo.
cmd /c "exit 1"
GOTO :EOF
Joe
  • 122,218
  • 32
  • 205
  • 338
  • 2
    You solution have big flaw. I can run interactive `CMD` session from PowerShell. And then invoke your script from interactive `CMD` session, which, as I understand, is expected usage scenario. And your script will refuse to work, just because interactive `CMD` session was started from PowerShell. – user4003407 Nov 25 '18 at 04:08
  • 1
    @PetSerAl - my use case doesn't require it to be 100% robust. In the case of a CMD session created from within PowerShell, it's OK to have a "false positive" in PowerShell detection. What I don't want is a "false negative", which would cause a user to wonder "why didn't my environment variables get set as I expected". – Joe Nov 25 '18 at 09:43
2

I did something like this for Chocolatey's RefreshEnv.cmd script: Make refreshenv.bat error if powershell.exe is being used.

My solution didn't end being used, for unrelated reasons, but it's available in this repo: beatcracker/detect-batch-subshell. Here is copy of it, just in case.


Script that will only run if called directly from interactive command processor session

Script will detect if it's run from non-interactive session (cmd.exe /c detect-batch-subshell.cmd) and show approriate error message.

Non-interactive shell includes PowerShell/PowerShell ISE, Explorer, etc... Basically anything that will try to execute script by running it in the separate cmd.exe instance.

Hovewer, dropping into the cmd.exe session from PowerShell/PowerShell ISE and executing script there will work.

Dependencies

  • wmic.exe - comes with Windows XP Professional and up.

Example:

  1. Open cmd.exe
  2. Type detect-batch-subshell.cmd

Output:

> detect-batch-subshell.cmd

Running interactively in cmd.exe session.

Example:

  1. Open powershell.exe
  2. Type detect-batch-subshell.cmd

Output:

PS > detect-batch-subshell.cmd

detect-batch-subshell.cmd only works if run directly from cmd.exe!

Code

  • detect-batch-subshell.cmd
@echo off

setlocal EnableDelayedExpansion

:: Dequote path to command processor and this script path
set ScriptPath=%~0
set CmdPath=%COMSPEC:"=%

:: Get command processor filename and filename with extension
for %%c in (!CmdPath!) do (
    set CmdExeName=%%~nxc
    set CmdName=%%~nc
)

:: Get this process' PID
:: Adapted from: http://www.dostips.com/forum/viewtopic.php?p=22675#p22675
set "uid="
for /l %%i in (1 1 128) do (
    set /a "bit=!random!&1"
    set "uid=!uid!!bit!"
)

for /f "tokens=2 delims==" %%i in (
    'wmic Process WHERE "Name='!CmdExeName!' AND CommandLine LIKE '%%!uid!%%'" GET ParentProcessID /value'
) do (
    rem Get commandline of parent
    for /f "tokens=1,2,*" %%j in (
        'wmic Process WHERE "Handle='%%i'" GET CommandLine /value'
    ) do (

        rem Strip extra CR's from wmic output
        rem http://www.dostips.com/forum/viewtopic.php?t=4266
        for /f "delims=" %%x in ("%%l") do (
            rem Dequote path to batch file, if any (3rd argument)
            set ParentScriptPath=%%x
            set ParentScriptPath=!ParentScriptPath:"=!
        )

        rem Get parent process path
        for /f "tokens=2 delims==" %%y in ("%%j") do (
            rem Dequote parent path
            set ParentPath=%%y
            set ParentPath=!ParentPath:"=!

            rem Handle different invocations: C:\Windows\system32\cmd.exe , cmd.exe , cmd
            for %%p in (!CmdPath! !CmdExeName! !CmdName!) do (
                if !ParentPath!==%%p set IsCmdParent=1
            )

            rem Check if we're running in cmd.exe with /c switch and this script path as argument
            if !IsCmdParent!==1 if %%k==/c if "!ParentScriptPath!"=="%ScriptPath%" set IsExternal=1
        )
    )
)

if !IsExternal!==1 (
    echo %~nx0 only works if run directly from !CmdExeName!^^!
    exit 1
) else (
     echo Running interactively in !CmdExeName! session.
 )

endlocal
beatcracker
  • 6,714
  • 1
  • 18
  • 41
  • Consider following PowerShell command line: ``cmd /c detect-batch-subshell.cmd `& using updated environment``. I am passing multiple commands to `cmd`, so that I update environment and use it in the same `cmd` instance. Is it valid usage? – user4003407 Nov 26 '18 at 02:59
  • @PetSerAl, You can bypass it by prepending `@` to the script path: `cmd /c @detect-batch-subshell.cmd & UsingUpdatedEnvironment`. Anyway this solution has some flaws, for example it fails if the path to the script contains spaces, Also it obtains the command line by using the slow and complex method of `WMI` where it could be easily obtained by `CMDCMDLINE `. I will add a much more simpler answer addressing those. – sst Nov 26 '18 at 03:24
  • Looks interesting, I'll take a look when I get time though it will take a bit of studying to understand the script – Joe Nov 26 '18 at 17:16
  • 1
    @sst Thanks for spotting spaces issue. Regarding `CMDCMDLINE`, you're absolutely right, it's much better then WMI, I just wasn't aware of it. – beatcracker Nov 26 '18 at 20:52
  • @PetSerAl The side effect of fixing issue with spaces in path is that if script is passed any parameters, it will not detect that it's launched in a subshell. So this should work :). – beatcracker Nov 26 '18 at 21:18
2

Like the answer from beatcracker I think it would be better to not take assumptions about the external shell that can be used to launch the batch script, for instance, the issue can also arise when running the batch file through the bash shell.

Because it exclusively uses the native facilities of CMD and has no dependency on any external tool or the WMI, the execution time is very fast.

@echo off
call :IsInvokedInternally && (
    echo Script is launched from an interactive CMD shell or from another batch script.
) || (
    echo Script is invoked by an external App. [PowerShell, BASH, Explorer, CMD /C, ...]
)
exit /b

:IsInvokedInternally
setlocal EnableDelayedExpansion
:: Getting substrings from the special variable CMDCMDLINE,
:: will modify the actual Command Line value of the CMD Process!
:: So it should be saved in to another variable before applying substring operations.
:: Removing consecutive double quotes eg. %systemRoot%\system32\cmd.exe /c ""script.bat""
set "SavedCmdLine=!cmdcmdline!"
set "SavedCmdLine=!SavedCmdLine:""="!"
set /a "DoLoop=1, IsExternal=0"
set "IsCommand="
for %%A in (!SavedCmdLine!) do if defined DoLoop (
    if not defined IsCommand (
        REM Searching for /C switch, everything after that, is CMD commands
        if /i "%%A"=="/C" (
            set "IsCommand=1"
        ) else if /i "%%A"=="/K" (
            REM Invoking the script with /K switch creates an interactive CMD session
            REM So it will be considered an internal invocatoin
            set "DoLoop="
        )
    ) else (
        REM Only check the first command token to see if it references this script
        set "DoLoop="

        REM Turning delayed expansion off to prevent corruption of file paths
        REM which may contain the Exclamation Point (!)
        REM It is safe to do a SETLOCAL here because the we have disabled the Loop,
        REM and the routine will be terminated afterwards.
        setlocal DisableDelayedExpansion
        if /i "%%~fA"=="%~f0" (
            set "IsExternal=1"
        ) else if /i "%%~fA"=="%~dpn0" (
            set "IsExternal=1"
        )
    )
)
:: A non-zero ErrorLevel means the script is not launched from within CMD.
exit /b %IsExternal%

It checks the command line that used to launch the CMD shell to tell if script have been launched from within CMD or by an external app using the command line signature /C script.bat which is typically used by non CMD shells to launch batch scripts.

If for any reason the external launch detection needs to bypasses, for instance when manually launching the script with additional commands to take advantage the defined variables, it can done by prepending @ to the path of the script in CMD command line:

cmd /c @MyScript.bat & AdditionalCommands
sst
  • 1,443
  • 1
  • 12
  • 15
  • `prepending @ to the path` - could you elaborate why this works? My batch is a bit rusty. – beatcracker Nov 26 '18 at 20:55
  • @beatcracker, well this solution will not detect every possible method of batch script invocation through the command line, it will just checks the most common way of doing it which is currently in use by `Powershell` or the windows shell component (e.g Explorer.exe) or other programs. For instance, invoking the batch file this way: `cmd /s /c MyScript.bat` breaks the pattern that this or your solution is relying upon. Prepending the `@` to the batch file path is the simplest and safest method of breaking the pattern. – sst Nov 26 '18 at 21:37
  • @beatcracker, But still it can detect the external invocation even if one passes additional parameters to batch script provided that the pattern `PathToCMD /C PathToBatchScript` is preserved. for example this works(detects external invocation): `cmd /c ""MyScript.bat" some additional quoted or unquoted parameters"` – sst Nov 26 '18 at 21:52
  • 1
    Updated the script to cover more forms of invocation, like when other switches other than `/C` are also passed to CMD. e.g `cmd /d /s /c "script.cmd"` – sst Nov 28 '18 at 05:20