win32 GetConsoleMode() error code 6

Question

I'm following the example from a another post here Reading a password from std::cin where you grab the handle to standard input, get the current console mode, change the mode to stifle echo on input. For some reason, when i call GetStdHandle() it returns a valid handle, but when I invoke GetConsoleMode() it fails and returns error code 6. I'm using this in a cmake project. Are there any debug flags I am supposed to set for this to work as intended? Anyone else encountered this?

void set_stdin_echo(bool enable) {
    HANDLE hStdin = GetStdHandle(STD_INPUT_HANDLE);
    if (hStdin == INVALID_HANDLE_VALUE) {
        std::cout << "invalid handle " << GetLastError() << std::endl;
        return;
    }
    else if (hStdin == NULL) {
        std::cout << "no handle associated" << std::endl;
        return;
    }

    DWORD mode = 0;
    if (GetConsoleMode(hStdin, &mode) == 0) {
        std::cout << "Could not get console mode" << GetLastError() << std::endl;
    }

    std::cout << mode << std::endl;
    if (!enable)
        mode &= ~ENABLE_ECHO_INPUT;
    else
        mode |= ENABLE_ECHO_INPUT;
    std::cout << mode << std::endl;

    if (SetConsoleMode(hStdin, mode) == 0) {
        std::cout << "Could not set input mode" << std::endl;
        std::cout << GetLastError() << std::endl;
    }
}

EDIT: reproduced with this code.

CMakeLists.txt

cmake_minimum_required (VERSION 2.6)
project(tester)

if(CMAKE_COMPILER_IS_GNUCXX)
    set(CMAKE_CXX_FLAGS "-g -w -Wall -pedantic-errors -std=c++11")
elseif(MSVC)
    set(CMAKE_CXX_FLAGS "/EHsc")
    set(CMAKE_CXX_FLAGS_DEBUG "/EHsc /MTd")
    set(CMAKE_CXX_FLAGS_RELEASE "/EHsc /MT")
endif()

set(SOURCE_FILES "main.cpp")

add_executable(tester ${SOURCE_FILES})

main.cpp

#include <iostream>
#include <string>
#include <windows.h>

void set_stdin_echo(bool enable) {
    DWORD error;
    HANDLE hStdin = GetStdHandle(STD_INPUT_HANDLE);
    error = GetLastError();
    if (hStdin == INVALID_HANDLE_VALUE) {
        std::cout << "invalid handle " << error << std::endl;
        return;
    }
    else if (hStdin == NULL) {
        std::cout << "no handle associated" << std::endl;
        return;
    }

    DWORD mode = 0;
    if (GetConsoleMode(hStdin, &mode) == 0) {
        error = GetLastError();
        std::cout << "Could not get console mode" << error << std::endl;
    }

    std::cout << mode << std::endl;
    if (!enable)
        mode &= ~ENABLE_ECHO_INPUT;
    else
        mode |= ENABLE_ECHO_INPUT;
    std::cout << mode << std::endl;

    if (SetConsoleMode(hStdin, mode) == 0) {
        error = GetLastError();
        std::cout << "Could not set input mode" << std::endl;
        std::cout << error << std::endl;
    }
}

int main() {
    std::string input;
    std::cout << "password: ";
    set_stdin_echo(false);
    std::cin >> input;
    set_stdin_echo(true);

    return 0;
}

Visual Studio 12. Invoke these in some directory. I get "Could not get console mode 6", implying GetLastError() returns 6.

mkdir build
cmake ..
msbuild.exe tester.sln
./Debug/tester.exe

EDIT: It appears that this only fails when run from the command line. Within visual studio from the visual studio debugger, it succeeds. Still not enough knowledge of visual studio to determine what is going on. I just want to make a simple exe that can run from the command line and hide its input.

EDIT: cleanliness and also obtaining GetLastError asap after calls. Still no change.

Answer 1

This error 6 means "invalid handle" and means that the handle you have passed to GetConsoleMode is not a console handle. I.e. the STDIN has been redirected from somewhere.

To ensure you have the handle to the current console, regardless of whether STDIN is redirected from somewhere else, use CreateFile with the special name "CONIN$", which opens the console (if present).

Example:

HANDLE hConsole = CreateFile("CONIN$",  
    GENERIC_READ, FILE_SHARE_READ|FILE_SHARE_WRITE, 
    NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);