ANSI color Set Graphics Rendition breaking mid-batch and working after it proceeds

Question

I've got a batch that has a sub-section which iterates through lines of a file for EXEs to try running, and then the batch sorts the EXEs based on their exit codes.

For some reason, the ANSI SGR seems to break or echo the literal text after setting the graphics rendition of the previous one instead of re-rendering it.

I went back to re-reference this question and the original documentation, but I'm not sure why this specific area of my batches is mangling the ANSI coloration inside the console after the first line is echoed.

I swapped my tool with just Notepad, which you can manually close for a zero-exit, or use the Control Panel to end the process to get a non-zero exit.

The contents of test_map.log shouldn't matter too much since you're actually using just Notepad and sending it some args. This is what mine are set to:

C:\temp\qt_selftest.exe
C:\temp\sub_test.exe
C:\temp\cmd_module_test.exe
C:\temp\failing_qt_test.exe
C:\temp\passing_qt_test.exe
C:\temp\random_qt_test.exe
C:\temp\fail_module.exe
C:\temp\pass_module.exe

And as you can see from the screenshot, the lines are being treated literally. Within the actual batch that I pulled this from, it does go back to working again... but within that block and only in that block, it's broken.

Any idea where I might be messing this up?

I can't share the code directly due to ESC sequences being converted, so here is the gist: https://gist.github.com/the-nose-knows/1bebce2719e020188c6307cff736f951

If you need to re-add them before the [, use the alt-code of 027, as alt 0 2 7

Answer 1

As erykson said, this can be solved by making sure Virtual Terminal Mode is enabled. If you only care about PowerShell's color, you can add the /A switch to your call to CMD.exe, otherwise you're going to want a small process that handles this kind of like a shim, but one that also makes sure VTM is enabled. This isn't wholly a bad thing, as this abstraction layer could come in handy for future use cases and bugs.

The only "weird" part of this snippet would be my rel-path usage. This snippet is a process shim from a sub-directory to run a batch shim from up one directory.

The important parts of the code are the headers to include and enabling VTM for StdOut after getting the console handle.

#include <iostream>
#include <string>
#include <vector>
#include <iterator>
#include <Windows.h>

#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
#endif

bool uses_whitespace(std::string test_string)
{
    size_t path_white_space_query = test_string.find(' ', 0);

    if (path_white_space_query != std::string::npos)
    {
        return true;
    }

    return false;
}

int main(int argc, char* argv[])
{
    std::string this_app_path = std::string(argv[0]);

    auto it = this_app_path.find_last_of("\\", std::string::npos);

    std::string path(this_app_path, 0, it);

    // Just forwarding the args that were sent to this shim to a batch in a known location,
    // making sure whitespace arguments keep their quotes when forwarded.

    // CMD.exe will need proper quote-handling, or the call will get mangled.
    std::string str = "C:\\Windows\\System32\\cmd.exe /C \"\"" + path + "\\..\\app_shim.bat\"";
    std::vector<std::string> args;
    std::copy(argv + 1, argv + argc, std::back_inserter(args));

    for (auto const& arg : args)
    {
        if (uses_whitespace(arg))
        {
            str += (" \"" + arg + "\"");
        }
        else
        {
            str += (" " + arg);
        }
    }

    // end-of-CMD-call final wrapping quote
    str += "\"";

    HANDLE stdOutHandle = GetStdHandle(STD_OUTPUT_HANDLE);

    DWORD mode = 0;
    GetConsoleMode(stdOutHandle, &mode);
    SetConsoleMode(stdOutHandle, mode | ENABLE_VIRTUAL_TERMINAL_PROCESSING);

    int exit_code = system(str.c_str());

    CloseHandle(stdOutHandle);

    return exit_code;
}