Redirecting cout to a console in windows

Question

I have an application which is a relatively old. Through some minor changes, it builds nearly perfectly with Visual C++ 2008. One thing that I've noticed is that my "debug console" isn't quite working right. Basically in the past, I've use AllocConsole() to create a console for my debug output to go to. Then I would use freopen to redirect stdout to it. This worked perfectly with both C and C++ style IO.

Now, it seems that it will only work with C style IO. What is the proper way to redirect things like cout to a console allocated with AllocConsole()?

Here's the code which used to work:

if(AllocConsole()) {
    freopen("CONOUT$", "wt", stdout);
    SetConsoleTitle("Debug Console");
    SetConsoleTextAttribute(GetStdHandle(STD_OUTPUT_HANDLE), FOREGROUND_GREEN | FOREGROUND_BLUE | FOREGROUND_RED);
}

EDIT: one thing which occurred to me is that I could make a custom streambuf whose overflow method writes using C style IO and replace std::cout's default stream buffer with it. But that seems like a cop-out. Is there a proper way to do this in 2008? Or is this perhaps something that MS overlooked?

EDIT2: OK, so I've made an implementaiton of the idea I spelled out above. Basically it looks like this:

class outbuf : public std::streambuf {
public:
    outbuf() {
        setp(0, 0);
    }

    virtual int_type overflow(int_type c = traits_type::eof()) {
        return fputc(c, stdout) == EOF ? traits_type::eof() : c;
    }
};

int APIENTRY WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPTSTR lpCmdLine, int nCmdShow) {
    // create the console
    if(AllocConsole()) {
        freopen("CONOUT$", "w", stdout);
        SetConsoleTitle("Debug Console");
        SetConsoleTextAttribute(GetStdHandle(STD_OUTPUT_HANDLE), FOREGROUND_GREEN | FOREGROUND_BLUE | FOREGROUND_RED);  
    }

    // set std::cout to use my custom streambuf
    outbuf ob;
    std::streambuf *sb = std::cout.rdbuf(&ob);

    // do some work here

    // make sure to restore the original so we don't get a crash on close!
    std::cout.rdbuf(sb);
    return 0;
}

Anyone have a better/cleaner solution than just forcing std::cout to be a glorified fputc?

Answer 1

Updated Feb 2018:

Here is the latest version of a function which fixes this problem:

void BindCrtHandlesToStdHandles(bool bindStdIn, bool bindStdOut, bool bindStdErr)
{
    // Re-initialize the C runtime "FILE" handles with clean handles bound to "nul". We do this because it has been
    // observed that the file number of our standard handle file objects can be assigned internally to a value of -2
    // when not bound to a valid target, which represents some kind of unknown internal invalid state. In this state our
    // call to "_dup2" fails, as it specifically tests to ensure that the target file number isn't equal to this value
    // before allowing the operation to continue. We can resolve this issue by first "re-opening" the target files to
    // use the "nul" device, which will place them into a valid state, after which we can redirect them to our target
    // using the "_dup2" function.
    if (bindStdIn)
    {
        FILE* dummyFile;
        freopen_s(&dummyFile, "nul", "r", stdin);
    }
    if (bindStdOut)
    {
        FILE* dummyFile;
        freopen_s(&dummyFile, "nul", "w", stdout);
    }
    if (bindStdErr)
    {
        FILE* dummyFile;
        freopen_s(&dummyFile, "nul", "w", stderr);
    }

    // Redirect unbuffered stdin from the current standard input handle
    if (bindStdIn)
    {
        HANDLE stdHandle = GetStdHandle(STD_INPUT_HANDLE);
        if(stdHandle != INVALID_HANDLE_VALUE)
        {
            int fileDescriptor = _open_osfhandle((intptr_t)stdHandle, _O_TEXT);
            if(fileDescriptor != -1)
            {
                FILE* file = _fdopen(fileDescriptor, "r");
                if(file != NULL)
                {
                    int dup2Result = _dup2(_fileno(file), _fileno(stdin));
                    if (dup2Result == 0)
                    {
                        setvbuf(stdin, NULL, _IONBF, 0);
                    }
                }
            }
        }
    }

    // Redirect unbuffered stdout to the current standard output handle
    if (bindStdOut)
    {
        HANDLE stdHandle = GetStdHandle(STD_OUTPUT_HANDLE);
        if(stdHandle != INVALID_HANDLE_VALUE)
        {
            int fileDescriptor = _open_osfhandle((intptr_t)stdHandle, _O_TEXT);
            if(fileDescriptor != -1)
            {
                FILE* file = _fdopen(fileDescriptor, "w");
                if(file != NULL)
                {
                    int dup2Result = _dup2(_fileno(file), _fileno(stdout));
                    if (dup2Result == 0)
                    {
                        setvbuf(stdout, NULL, _IONBF, 0);
                    }
                }
            }
        }
    }

    // Redirect unbuffered stderr to the current standard error handle
    if (bindStdErr)
    {
        HANDLE stdHandle = GetStdHandle(STD_ERROR_HANDLE);
        if(stdHandle != INVALID_HANDLE_VALUE)
        {
            int fileDescriptor = _open_osfhandle((intptr_t)stdHandle, _O_TEXT);
            if(fileDescriptor != -1)
            {
                FILE* file = _fdopen(fileDescriptor, "w");
                if(file != NULL)
                {
                    int dup2Result = _dup2(_fileno(file), _fileno(stderr));
                    if (dup2Result == 0)
                    {
                        setvbuf(stderr, NULL, _IONBF, 0);
                    }
                }
            }
        }
    }

    // Clear the error state for each of the C++ standard stream objects. We need to do this, as attempts to access the
    // standard streams before they refer to a valid target will cause the iostream objects to enter an error state. In
    // versions of Visual Studio after 2005, this seems to always occur during startup regardless of whether anything
    // has been read from or written to the targets or not.
    if (bindStdIn)
    {
        std::wcin.clear();
        std::cin.clear();
    }
    if (bindStdOut)
    {
        std::wcout.clear();
        std::cout.clear();
    }
    if (bindStdErr)
    {
        std::wcerr.clear();
        std::cerr.clear();
    }
}

In order to define this function, you'll need the following set of includes:

#include <windows.h>
#include <io.h>
#include <fcntl.h>
#include <iostream>

In a nutshell, this function synchronizes the C/C++ runtime standard input/output/error handles with the current standard handles associated with the Win32 process. As mentioned in the documentation, AllocConsole changes these process handles for us, so all that's required is to call this function after AllocConsole to update the runtime handles, otherwise we'll be left with the handles that were latched when the runtime was initialized. Basic usage is as follows:

// Allocate a console window for this process
AllocConsole();

// Update the C/C++ runtime standard input, output, and error targets to use the console window
BindCrtHandlesToStdHandles(true, true, true);

This function has gone through several revisions, so check the edits to this answer if you're interested in historical information or alternatives. The current answer is the best solution to this problem however, giving the most flexibility and working on any Visual Studio version.

Answer 2

I'm posting a portable solution in answer form so it can be accepted. Basically I replaced cout's streambuf with one that is implemented using c file I/O which does end up being redirected. Thanks to everyone for your input.

class outbuf : public std::streambuf {
public:
    outbuf() {
        setp(0, 0);
    }

    virtual int_type overflow(int_type c = traits_type::eof()) {
        return fputc(c, stdout) == EOF ? traits_type::eof() : c;
    }
};

int APIENTRY WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPTSTR lpCmdLine, int nCmdShow) {
    // create the console
    if(AllocConsole()) {
        freopen("CONOUT$", "w", stdout);
        SetConsoleTitle("Debug Console");
        SetConsoleTextAttribute(GetStdHandle(STD_OUTPUT_HANDLE), FOREGROUND_GREEN | FOREGROUND_BLUE | FOREGROUND_RED);  
    }

    // set std::cout to use my custom streambuf
    outbuf ob;
    std::streambuf *sb = std::cout.rdbuf(&ob);

    // do some work here

    // make sure to restore the original so we don't get a crash on close!
    std::cout.rdbuf(sb);
    return 0;
}

Answer 3

If console is for debug only, you can just use OutputDebugStringA/OutputDebugStringW functions. Their output directed to Output window in VS if you are in debug mode, otherwise you can use DebugView to see it.

Answer 4

This works with VC++ 2017 for c++ style I/O

AllocConsole();

// use static for scope
static ofstream conout("CONOUT$", ios::out); 
// Set std::cout stream buffer to conout's buffer (aka redirect/fdreopen)
cout.rdbuf(conout.rdbuf());

cout << "Hello World" << endl;

Answer 5

For the original you could just use sync_with_stdio(1) example:

if(AllocConsole())
{
    freopen("CONOUT$", "wt", stdout);
    freopen("CONIN$", "rt", stdin);
    SetConsoleTitle(L"Debug Console");
    std::ios::sync_with_stdio(1);
}

Answer 6

The ios library has a function that lets you re-sync C++ IO to whatever the standard C IO is using: ios::sync_with_stdio().

There's a nice explanation here: http://dslweb.nwnexus.com/~ast/dload/guicon.htm.

Answer 7

From what I can tell, your code should work with VC 2005, if it's your first activity with the console.

After checking a few possibilities, you might be trying to write something before you allocate the console. Writing to std::cout or std::wcout at that point will fail and you need to clear the error flags before making further output.

Answer 8

Raymond Martineau makes a good point about it being 'the first thing you do'.

I had a redirection problem, which I forget the details of now, where it turned out that very early in the execution of the app, the runtime makes some decisions about output directions which then last for the rest of the application.

After following this through the CRT source, I was able to subvert this mechanism by clearing a variable within the CRT, which made it take another look at things once I'd done my AllocConsole.

Obviously this sort of stuff is not going to be portable, possibly even across toolchain versions, but it might help you out.

After your AllocConsole, step all the way down into the next cout output and find out where it's going and why.

Answer 9

Try this 2 liner:

    AllocConsole(); //debug console
    std::freopen_s((FILE**)stdout, "CONOUT$", "w", stdout); //just works

Answer 10

I don't know, but as to why this is happening, freopen("CONOUT$", "w", stdout); might not redirect the stdout handle in the process parameter block (NtCurrentPeb()->ProcessParameters->StandardOutput) to whatever the LPC call to CSRSS/Conhost returns in response to a request for the stdout handle for the attached console of the process (NtCurrentPeb()->ProcessParameters->ConsoleHandle). It might just make the LPC call and then assign the handle to the FILE * stdout global variable. C++ cout doesn't use FILE * stdout at all, and probably still might not sync with the PEB for the standard handles.

Answer 11

I am not sure I understand the problem completely but if you want to be able to simply spit out data to console for diagnostic purpose.. why dont you try out System::Diagnostics::Process::Execute() method or some method in that namespace??

Apologies in advance if it was irrelevant