Inconsistent string encryption result between program and Shell

Question

I've wrote the below code snippet that passes a string to openssl by echo'ing, using popen; however the result is different than if I manually pass the arguments in PowerShell. Any help is appreciated...

void encrypt_message(const char * message_in, char * encrypted_return)
{
    char encryption_cmd[1024] = { '\0' };
    strcat(encryption_cmd, "echo '");
    strcat(encryption_cmd, message_in);
    strcat(encryption_cmd, "' | openssl.exe dgst -sha256 -hmac ");
    strcat(encryption_cmd, SEC_KEY);

    printf("CMD= %s\n",encryption_cmd);

    char   psBuffer[1024] = { '\0' };
    FILE   *pPipe;

    if ((pPipe = _popen(encryption_cmd, "rt")) == NULL)
        return;

    while (fgets(psBuffer, 1024, pPipe));
        
    if (!feof(pPipe))
        printf("Error: Failed to read the pipe to the end.\n");

    sscanf(psBuffer, "(stdin)= %s", encrypted_return);

    printf("RESULT= %s\n",encrypted_return);
}

This is the output of the function:

However when I manually invoke in PowerShell I get a different result

It looks as if popen is sprinkling something in that openssl is interpretting, yielding a different result. But I can't narrow what exactly! Thanks

UPDATE: I get the same result as my program code when I pass arguments in cmd.exe instead of PowerShell.... so it appears it's PowerShell that is doing something strange.

Answer 1

echo 'this' does different things in cmd.exe vs. PowerShell:

• In cmd.exe, the command output is verbatim string 'this' , including the single quotes and the trailing space, plus a trailing newline.

  • Unfortunately, cmd.exe's echo echoes any quoting as specified; generally cmd.exe does not understand single-quoting ('...'); while it does understand double-quoting ("..."), and while this double-quoting is necessary to protect metacharacters such as &, echo still echoes them (instead of stripping them, which you'd expect for quotes that syntactic function).

• In PowerShell, where echo is simply a built-in alias for the Write-Output cmdlet, the command output is verbatim string this - the syntactic quotes are stripped, and the trailing space is irrelevant - plus a trailing newline.

  • PowerShell understands both '...' strings (verbatim) and "..." (expandable, i.e. interpolating), and properly strips quotes with syntactic function; for more information about string literals in PowerShell, see the bottom section of this answer.

There is no single echo-based command that would work the same in both shells.

If it's acceptable to append a trailing newline:

As stated, in effect, echo invariably results in a trailing newline getting appended to the string being echoed.

• In cmd.exe, pass the string unquoted and immediately follow it with |:

  • echo this| openssl.exe ...

  • if the string contains cmd.exe metacharacters, such as & or ;, ^-escape them.

• In PowerShell, while echo 'this' works, you can simply omit the echo and rely on PowerShell's implicit output behavior:

  • 'this' | openssl.exe ...
  • If the string has embedded ' chars., escape them as ''.

If it's not acceptable to append a trailing newline:

• In cmd.exe, you have to resort to a hack:

  • <NUL set /p ="this" | openssl dgst ...
  • See this answer for more information and the limitations on the limitations on what strings can be output this way; in short: the string mustn't start with =, must not contain embedded " chars., and must not have leading tabs and/or spaces, which are invariably stripped.

• In PowerShell, as of v7.1, you currently cannot send a string via the pipeline to an external program without a trailing newline - see GitHub issue #5974.

  • The workaround is to call cmd.exe and use its hack:
    • cmd /c '<NUL set /p ="this" | openssl dgst ...'
  • Again, be sure to escape embedded ' as ''; if you want to use an expandable string ("...") so you can use string interpolation to embed PowerShell variables, escape the embedded " as `":
    • $str = 'this'; cmd /c "<NUL set /p =`"$str`" | openssl dgst ..."