c win32 api - GetStdHandle(STD_OUTPUT_HANDLE) is invalid, very perplexing

Question

I have the following fragment in my WinMain and I am launching this GUI app from the console. I want to redirect output to the console from which my app was launched.I am getting the "The handle is invalid." error after GetStdHandle().

However, if I use AllocConsole instead of AttachConsole, it works fine. In addition, if I use STD_ERROR_HANDLE instead of STD_OUTPUTHANDLE then fprintf(stderr, "errror") works fine.

I saw a blog entry which had the same problem but no solution. I am using vc 2010 compiler on 64 bit windows 7.

Thanks!

bConsole = AttachConsole(ATTACH_PARENT_PROCESS) != FALSE;

if (bConsole)
{
    int fd = 0;
    long lStdOut;
    lStdOut = (long)GetStdHandle(STD_OUTPUT_HANDLE);
    fd = _open_osfhandle(lStdOut, _O_TEXT);
    if (fd > 0)
    {
        *stdout = *_fdopen(fd, "w");
        setvbuf(stdout, NULL, _IONBF, 0 );
    }
}
printf("Test!!!!!!!!!!!!");

You need to check for errors after calling `GetStdHandle`. If it fails it will return `INVALID_HANDLE_VALUE`. Check for this and the call `GetLastError` if necessary. Then tell us what `GetLastError` reports. — David Heffernan, Feb 25 '11 at 11:43
You should define `lStdOut` as `intptr_t lStdOut;` instead of using `long lStdOut;`. See http://msdn.microsoft.com/en-us/library/bdts1c9x.aspx — Oleg, Feb 25 '11 at 13:11
@Oleg Not true, it's a `HANDLE` – see [GetStdHandle](http://msdn.microsoft.com/en-us/library/ms683231%28VS.85%29.aspx) — David Heffernan, Feb 25 '11 at 13:58
@David Heffernan: "The `_open_osfhandle()` function allocates a C run-time file handle and sets it to point to the operating-system file handle." (see http://support.microsoft.com/kb/185727) or "Associates a C run-time file descriptor with an existing operating-system file handle" (see http://msdn.microsoft.com/en-us/library/bdts1c9x(VS.80).aspx). The type `intptr_t` is the representation of `HANDLE`. Just open `` and search for `intptr_t`. You will find many examples. Moreover the link http://msdn.microsoft.com/en-us/library/bdts1c9x.aspx shows **exact prototype** of `_open_osfhandle`. — Oleg, Feb 25 '11 at 14:12
@Oleg `GetStdHandle` returns a `HANDLE`. It so happens that, at the current time, that is represented by `intptr_t`. However, it is correct to use `HANDLE` in this instance. — David Heffernan, Feb 25 '11 at 14:14
@David Heffernan: Of course `GetStdHandle` returns a `HANDLE`, but `_open_osfhandle` use `intptr_t` as input parameter. One have to make type casting, but in the code of the question one uses `long` instead of `intptr_t`, but the size of `long` can be different from the size of `HANDLE` or `intptr_t` (having the same size as `HANDLE`). — Oleg, Feb 25 '11 at 14:20
@Oleg OK, I see. I'd cast it on the way in to `_open_osfhandle` I suppose, but it does come to the same thing I guess. — David Heffernan, Feb 25 '11 at 14:30
possible duplicate of [Where do writes to stdout go when launched from a cygwin shell, no redirection](http://stackoverflow.com/questions/4028353/where-do-writes-to-stdout-go-when-launched-from-a-cygwin-shell-no-redirection) — Ben Voigt, Feb 25 '11 at 18:42

score 7 · Answer 1 · edited May 23 '17 at 12:24

7

AttachConsole does associate your process with a console, but stdout has already been opened (and connected to the old handle, whatever it was).

Overwriting stdout directly is a terrible idea. Instead, you must freopen("CONOUT$", "w", stdout); to get stdout going to the console.

But there are a lot of other little details. Have a look at my question Where do writes to stdout go when launched from a cygwin shell, no redirection which covers your problem in the question, then asks a question about some corner cases. Finally there's a code sample which incorporates everything.

edited May 23 '17 at 12:24

Community

1
1

answered Feb 25 '11 at 18:53

Ben Voigt

277,958
43
419
720

@Ben his error appears to be long before he overwrites stdout. He says GetStdHandle fails. – David Heffernan Feb 25 '11 at 18:58
@David: Please go read my question, which has a complete, 100% working code sample and covers numerous corner cases (such as shell redirection to file) as well. – Ben Voigt Feb 25 '11 at 18:59
@Ben I was more interested in this question – David Heffernan Feb 25 '11 at 19:00
@Ben So I read the question you linked to, but it doesn't seem to apply here. Perhaps you should re-read this question. – David Heffernan Feb 25 '11 at 19:22
1

@David: This question says "I want to redirect output to the console from which my app was launched." Doing that is addressed in my question. I'd figured it out on my own for combinations other than cygwin bash, so my question was specifically about cygwin bash, but the code I posted there meets pug's requirements and then some. I also copied the single line which solves the specific problem pug is having into an answer here. Yes, `GetStdHandle` returns garbage after `AttachConsole`, so **you don't use it**. You use `freopen` and life is good. – Ben Voigt Feb 25 '11 at 19:28
@Ben `GetStdHandle` doesn't return garbage after `AttachConsole` when I do it. Perhaps he is using Cygwin. Perhaps its something else. What's more, the code you posted is bizarre. I can't understand why you would test `if (GetStdHandle(STD_OUTPUT_HANDLE))` as you do. That will always evaluate true! The error condition is when it returns `INVALID_HANDLE_VALUE` which also evaluates true. – David Heffernan Feb 25 '11 at 19:39
@David: ["If an application does not have associated standard handles, such as a service running on an interactive desktop, and has not redirected them, the return value is NULL."](http://msdn.microsoft.com/en-us/library/ms683231.aspx) I've never seen the function *fail*, but if you e.g. launched from explorer, the handle it returns is NULL, and that's what I'm checking for. I really did test that code in all the different cases listed in my question. – Ben Voigt Feb 25 '11 at 19:56
@David: I've made assumptions based on what I think the documentation should say, so I don't really have a leg to stand on. Maybe I should be checking for `INVALID_HANDLE_VALUE` as well, but if the process creation block can't be read I don't think the situation is recoverable. – Ben Voigt Feb 25 '11 at 20:22
@Ben Of course, it would all be a lot easier if @Pug were to contribute something. Kind of hard to debug somebody else's problem if they won't help! – David Heffernan Feb 25 '11 at 20:26

score 6 · Answer 2 · answered Apr 15 '11 at 12:42

A Windows Subsystem process (i.e. one with WinMain) will not have a STDOUT, STDERR or STDIN, unless it was specifically given one on launch. The assumption is that since it is a windows program you are interacting with it via Windows.

I.e. GetStdHandle is not returning a handle to STDOUT because you haven't got a STDOUT.

To give it one, launch it thus:

winprog.exe > output.txt 2>&1

If launched in that way, it will have both a STDOUT and STDERR, which will both go into the named file.

As pointed out by other users already, AttachConsole will give you a console (nearest unix/linux equivalent is a TTY) but it will not give you a STDOUT. If you want one you will have to set it as a separate step. If you want it to be the console, you can have that too.

On the other hand a Console subsystem program (one with main) will by default have a STDIN, STDOUT and STDERR all set to the Console. You can detach the process from the console, and close them if you want.

score 2 · Answer 3 · answered Feb 25 '11 at 18:11

2

I added the following code into the default Visual Studio C++ GUI project, right at the start of WinMain.

if (AttachConsole(ATTACH_PARENT_PROCESS))
{
    if (GetStdHandle(STD_OUTPUT_HANDLE) == INVALID_HANDLE_VALUE)
        MessageBox(0, L"Invalid Handle", NULL, 0);
    else
        MessageBox(0, L"Valid Handle", NULL, 0);
}

When I run the GUI program from the debugger, or from Explorer, no message box shows. In other words we can't attach a console. When I run from cmd I see the "Valid Handle" message.

I conclude that there is in fact no problem with this basic approach, but that something that you are not showing us is causing the problem.

answered Feb 25 '11 at 18:11

David Heffernan

601,492
42
1,072
1,490

For future readers who may not have read the entire comment thread below my answer, the problem with the test in David's snippet is that `GetStdHandle` doesn't use the magic number of `INVALID_HANDLE_VALUE`, instead it returns `NULL`. And `NULL != INVALID_HANDLE_VALUE`, so it wrongly passed the test. – Ben Voigt Aug 29 '16 at 19:14
@Ben Well it can return INVALID_HANDLE_VALUE or NULL according to the docs. But the test in my answer is bizarre. Probably best for me to delete the answer. – David Heffernan Aug 29 '16 at 20:02

score 0 · Answer 4 · answered Feb 25 '11 at 09:34

0

I think your problem is here:

long lStdOut;
lStdOut = (long)GetStdHandle(STD_OUTPUT_HANDLE);
fd = _open_osfhandle(lStdOut, _O_TEXT);

I'm not very good with Win32 API, but I think handles have their own HANDLE types, which are essentially pointers I think, and 64-bit on Win64. long type in Win64 is though still 32-bit for some reason.

This is the declaration from MSDN:

HANDLE WINAPI GetStdHandle(
  __in  DWORD nStdHandle
);

Declaration for _open_osfhandle:

int _open_osfhandle (
   intptr_t osfhandle,
   int flags 
);

answered Feb 25 '11 at 09:34

Maister

4,978
1
31
34

Actually first I tried this with HANDLE instead of long. It's later that I was trying different stuff to make this work that I changed it to long. However, the same code works if I change STD_OUTPUT_HANDLE to STD_ERROR_HANDLE so I am thinking that this may not be the problem. The reason this probably doesn't matter is because I am using the win32 compiler. – Ali Feb 25 '11 at 09:50
3

Maister makes a good point. It doesn't solve your problem, but you should use the right types. – David Heffernan Feb 25 '11 at 11:42

score 0 · Answer 5 · answered Jun 03 '19 at 18:53

To redirect output to Console use the following code:

AllocConsole();
hConsole = GetStdHandle(STD_OUTPUT_HANDLE);

then you can write to the Console using WriteFile().

    WriteFile(
        hConsole,                
        L"this is a debug line\n", 
        21, // string length 
        NULL,             // bytes written 
        NULL);

c win32 api - GetStdHandle(STD_OUTPUT_HANDLE) is invalid, very perplexing

5 Answers5