No console output when using AllocConsole and target architecture x86

Question

I have a WinForms project, and if the user want's a debug console, I allocate a console with AllocConsole().

All console output works normally with the target architecture set to "Any CPU", but when I change it to "x86" it doesn't output anything (Console.Read() still works as expected). If I open the EXE directly, the output works. It looks like Visual Studio redirects it into it's own "Output" window.

I also tried this answer, but it didn't work, I also tried Console.SetOut(GetStdHandle(-11)), which didn't work either.

Setting the target architecture to 'Any CPU' is no option for me.

So here are my two questions:

Why is this only the case when the target architecture is set to x86?
How can I output to my console when running inside of Visual Studio?

@HansPassant: No, there are no exceptions, but your comment helped me finding the (partial) solution: When i open the .exe directly, it works, but when i debug my solution all output goes to the "Output" view in VS. But two questions are still here: Why only with x86 and how can make console output when i debug my solution? (Still +1 for you :D) — teamalpha5441, Mar 24 '13 at 23:39

score 39 · Accepted Answer · edited Jan 21 '19 at 05:37

39

When "Enable native code debugging" is enabled, output from consoles crated with AllocConsole is redirected to the debug output window instead.

The reason this only happens in x86 and not AnyCPU is because you can only debug native code in an x86 application.

Note that this behavior only occurs with consoles created with AllocConsole. A console application's output is not redirected.

EDIT: The other reason for the console not outputting text is when you've written to the console before calling AllocConsole.

Regardless of the reason, this code will restore output if it was redirected, and reopen the console in case it's invalid. It uses the magic number 7 which is what the handle of stdout usually equals to.

using System;
using System.IO;
using System.Runtime.InteropServices;

public static class ConsoleHelper
{
    public static void CreateConsole()
    {
        AllocConsole();

        // stdout's handle seems to always be equal to 7
        IntPtr defaultStdout = new IntPtr(7);
        IntPtr currentStdout = GetStdHandle(StdOutputHandle);

        if (currentStdout != defaultStdout)
            // reset stdout
            SetStdHandle(StdOutputHandle, defaultStdout);

        // reopen stdout
        TextWriter writer = new StreamWriter(Console.OpenStandardOutput()) 
        { AutoFlush = true };
        Console.SetOut(writer);
    }

    // P/Invoke required:
    private const UInt32 StdOutputHandle = 0xFFFFFFF5;
    [DllImport("kernel32.dll")]
    private static extern IntPtr GetStdHandle(UInt32 nStdHandle);
    [DllImport("kernel32.dll")]
    private static extern void SetStdHandle(UInt32 nStdHandle, IntPtr handle);
    [DllImport("kernel32")]
    static extern bool AllocConsole();
}

See How to detect if Console.In (stdin) has been redirected? for another way to detect if the console handles have been redirected.

edited Jan 21 '19 at 05:37

Pierre Arnaud

10,212
11
77
108

answered Apr 11 '13 at 23:16

Stephen

430
6
6

Sounds great, but the weird thing is that the native code debugging is disabled in my project properties... – teamalpha5441 Apr 11 '13 at 23:45
Are you sure your startup project doesn't have it enabled? This is the only one that matters.I've added a solution that hopefully should work to my answer. – Stephen Apr 13 '13 at 15:03
Epic Stephen is epic! It works with your solution. I only skipped the currentStdOut != defaultStdOut and TextWriter part (also the redirect check for stdin doesn't work because stdin has worked everytime). Thank you for your great help! – teamalpha5441 Apr 14 '13 at 14:56
5

Thanks for the very nice answer. I would like to add that using the `0x7` magic number for the default stout handle is not very safe as it's not a real handle and probably it's an undocumented feature. You'd better use `CreateFile(L"CONOUT$")` to lookup the actual console screen buffer and test for that. – ceztko Oct 23 '14 at 22:41
2

In C++, the full instruction is `CreateFile(L"CONOUT$", GENERIC_READ | GENERIC_WRITE, FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0);`. Shouldn't be extremely hard to port it to C#. – ceztko Oct 23 '14 at 22:43
5

Note: The default STD out handle is not 7 on Windows Server 2012 R2. Seems the value is 1228 instead (note: Citrix is also installed on this server I'm using). If I force 7, nothing shows. – James Wilkins Dec 04 '15 at 01:09
Do not make assumptions about console handle values. Prior to Windows 8, console buffer handles were managed by the console itself; console I/O used an LPC port; and thus console handles were flagged for routing to special LPC-based functions by setting the lower 2 bits. In Windows 8+ the console uses regular NT I/O (e.g. `NtReadFile`) via the condrv.sys device driver, so console handles are regular kernel File handles opened on the ConDrv device. To ensure that `AllocConsole` updates the standard handles, reset them to `NULL` beforehand. This is documented behavior. – Eryk Sun Aug 16 '18 at 15:12
@ErykSun This code isn't working in Windows 10 (as you mentioned), basically, AllocConsole opens Console, but calling any method on the C# Console class causes an exception "The handle is invalid". What's the fix? – tunafish24 Oct 08 '19 at 10:03
1

@tunafish24, reset handles of interest to `NULL` via `SetStdHandle` before calling `AllocConsole`. Do not call `SetStdHandle` afterward. There's no need to make any assumption about the handle value. In Windows 8+, it will not be a legacy console pseudohandle (i.e. 3, 7, 11, 15, etc), but will instead be a kernel handle for a file on the ConDrv device (e.g. "\Device\ConDrv\Output") with an arbitrary handle value that's a multiple of 4 (e.g. 28, 112, 448). – Eryk Sun Oct 08 '19 at 12:58
@ErykSun I tried it but it still causes an exception. First I call SetStdHandle for Input, Output, Error and set it to IntPtr.Zero. Then AllocConsole, then C# Console class to Set Window size. The first time console is opened, it works fine, then I close it, then when I open it again, the Console.SetWindowSize throws exception - "The handle is invalid". If I comment out Console.SetWindowSize then it loads fine but I've noticed that the second time, console doesn't print out colored text, which I need because I'm using it for logging. – tunafish24 Oct 08 '19 at 13:43

Pavlo K · Answer 2 · 2018-02-19T16:42:30.687

None of the earlier answers worked well for me with VS2017 and Windows 10 (for instance they failed if launch app in debug mode).

Below you can find a little bit enhanced code. Idea is the same, but magic numbers are removed (Ceztko already mentioned that) and all necessary in\out streams are initialized.

This code works for me if create a new console (alwaysCreateNewConsole = true).

Attaching to console of parent process (alwaysCreateNewConsole = false) has several drawbacks. For example I was unable to completely mimic behavior of console app launched from cmd. And I'm not sure that it is possible at all.

And most important: after revision of Console class I reconsidered general idea of using Console class with manually created console. It works well (I hope) for most of the cases, but can bring a lot of pain in future.

    static class WinConsole
    {
        static public void Initialize(bool alwaysCreateNewConsole = true)
        {
            bool consoleAttached = true;
            if (alwaysCreateNewConsole
                || (AttachConsole(ATTACH_PARRENT) == 0
                && Marshal.GetLastWin32Error() != ERROR_ACCESS_DENIED))
            {
                consoleAttached = AllocConsole() != 0;
            }

            if (consoleAttached)
            {
                InitializeOutStream();
                InitializeInStream();
            }
        }

        private static void InitializeOutStream()
        {
            var fs = CreateFileStream("CONOUT$", GENERIC_WRITE, FILE_SHARE_WRITE, FileAccess.Write);
            if (fs != null)
            {
                var writer = new StreamWriter(fs) { AutoFlush = true };
                Console.SetOut(writer);
                Console.SetError(writer);
            }
        }

        private static void InitializeInStream()
        {
            var fs = CreateFileStream("CONIN$", GENERIC_READ, FILE_SHARE_READ, FileAccess.Read);
            if (fs != null)
            {
                Console.SetIn(new StreamReader(fs));
            }
        }

        private static FileStream CreateFileStream(string name, uint win32DesiredAccess, uint win32ShareMode,
                                FileAccess dotNetFileAccess)
        {
            var file = new SafeFileHandle(CreateFileW(name, win32DesiredAccess, win32ShareMode, IntPtr.Zero, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, IntPtr.Zero), true);
            if (!file.IsInvalid)
            {
                var fs = new FileStream(file, dotNetFileAccess);
                return fs;
            }
            return null;
        }

        #region Win API Functions and Constants
        [DllImport("kernel32.dll",
            EntryPoint = "AllocConsole",
            SetLastError = true,
            CharSet = CharSet.Auto,
            CallingConvention = CallingConvention.StdCall)]
        private static extern int AllocConsole();

        [DllImport("kernel32.dll",
            EntryPoint = "AttachConsole",
            SetLastError = true,
            CharSet = CharSet.Auto,
            CallingConvention = CallingConvention.StdCall)]
        private static extern UInt32 AttachConsole(UInt32 dwProcessId);

        [DllImport("kernel32.dll",
            EntryPoint = "CreateFileW",
            SetLastError = true,
            CharSet = CharSet.Auto,
            CallingConvention = CallingConvention.StdCall)]
        private static extern IntPtr CreateFileW(
              string lpFileName,
              UInt32 dwDesiredAccess,
              UInt32 dwShareMode,
              IntPtr lpSecurityAttributes,
              UInt32 dwCreationDisposition,
              UInt32 dwFlagsAndAttributes,
              IntPtr hTemplateFile
            );

        private const UInt32 GENERIC_WRITE = 0x40000000;
        private const UInt32 GENERIC_READ = 0x80000000;
        private const UInt32 FILE_SHARE_READ = 0x00000001;
        private const UInt32 FILE_SHARE_WRITE = 0x00000002;
        private const UInt32 OPEN_EXISTING = 0x00000003;
        private const UInt32 FILE_ATTRIBUTE_NORMAL = 0x80;
        private const UInt32 ERROR_ACCESS_DENIED = 5;

        private const UInt32 ATTACH_PARRENT = 0xFFFFFFFF;

        #endregion
    }

Not working for me in VS2017 (or VS2015 with disabled vhost). — ZorgoZ, Mar 07 '18 at 06:21
Works fine on W10 VS2017/VS2019, but when running on Windows 7 doesn't work. — Kevin Kouketsu, Apr 12 '19 at 16:42
that works on W10 / VS2019 !! (even if Console methods are WinApi and not StdCall and if most of the EntryPoint and CharSet are useless here...) — Yet Another Code Maker, Jul 08 '19 at 14:21
Love it. It's great. You really helped me out here. Thank you. ...........one small thing. Console.ForegroundColor doesn't work. I don't suppose any of you wizards could plug that in? — user875234, Jan 17 '20 at 04:49
As mentioned by @KevinKouketsu this hangs on Windows 7 on `new FileStream` for the `"CONIN$"`. The problem is the FileStream constructor which attempts synchronous reading from the Stream which leads to awaiting for user input. — Zverev Evgeniy, Mar 23 '23 at 10:02

score 6 · Answer 3 · answered Dec 09 '15 at 04:31

Following worked for me in vs 2015, none worked from other answers:

Source: https://social.msdn.microsoft.com/profile/dmitri567/?ws=usercard-mini

using System;   
using System.Windows.Forms;   
using System.Text;   
using System.IO;   
using System.Runtime.InteropServices;   
using Microsoft.Win32.SafeHandles;   

namespace WindowsApplication   
{   
    static class Program   
    {   
        [DllImport("kernel32.dll",   
            EntryPoint = "GetStdHandle",   
            SetLastError = true,   
            CharSet = CharSet.Auto,   
            CallingConvention = CallingConvention.StdCall)]   
        private static extern IntPtr GetStdHandle(int nStdHandle);   
        [DllImport("kernel32.dll",   
            EntryPoint = "AllocConsole",   
            SetLastError = true,   
            CharSet = CharSet.Auto,   
            CallingConvention = CallingConvention.StdCall)]   
        private static extern int AllocConsole();   
        private const int STD_OUTPUT_HANDLE = -11;   
        private const int MY_CODE_PAGE = 437;   

        static void Main(string[] args)   
        {   
            Console.WriteLine("This text you can see in debug output window.");   

            AllocConsole();   
            IntPtr stdHandle=GetStdHandle(STD_OUTPUT_HANDLE);   
            SafeFileHandle safeFileHandle = new SafeFileHandle(stdHandle, true);   
            FileStream fileStream = new FileStream(safeFileHandle, FileAccess.Write);   
            Encoding encoding = System.Text.Encoding.GetEncoding(MY_CODE_PAGE);   
            StreamWriter standardOutput = new StreamWriter(fileStream, encoding);   
            standardOutput.AutoFlush = true;   
            Console.SetOut(standardOutput);   

            Console.WriteLine("This text you can see in console window.");   

            MessageBox.Show("Now I'm happy!");   
        }   
    }   
}

Didn't work for me in VS2017- debug,anycpu (both msgs to debug window) — FastAl, Feb 11 '19 at 17:47

dss539 · Answer 4 · 2015-02-19T15:02:39.357

4

I also had this problem. Every time I tried to debug my app, the console was blank. Strangely, launching the exe without the debugger worked fine.

I found that I had to Enable the Visual Studio hosting process from the project's Debug menu.

Stephen is correct that Enable native code debugging does redirect the console to the Output window. However, regardless of the native code debugging setting, I saw absolutely no output in either place until I enabled the Visual Studio hosting process.

This could have been the reason that merely disabling native code debugging did not solve your issue.

edited Feb 19 '15 at 15:02

answered Feb 07 '15 at 02:12

dss539

6,804
2
34
64

2

This option was removed in Visual Studio 2017. – Demonslay335 Feb 08 '19 at 00:35

Senthilnathan PL · Answer 5 · 2019-08-16T05:18:37.397

Just wanted to post the answer from Visual studio Developer community. https://developercommunity.visualstudio.com/content/problem/12166/console-output-is-gone-in-vs2017-works-fine-when-d.html Go to this link and look at the answer from Ramkumar Ramesh. I have tested this code in VS 2017. I spent one day to find this answer. Hope it helps you as well.

Edit-- As suggessted by Mike to include some description. I would like to suggesst some corrections in Zuniar answer. He tested with VS 2015. But that would not work in VS 2017. Instead of GetStdHandle, Please use CreateFile reference from kernel32.dll

IntPtr stdHandle = CreateFile("CONOUT$", GENERIC_WRITE, FILE_SHARE_WRITE, 0, 
OPEN_EXISTING, 0, 0);

Before adding above code, please declare

[DllImport("kernel32.dll", SetLastError = true)]
private static extern IntPtr CreateFile(string lpFileName, uint 
dwDesiredAccess, uint dwShareMode, uint lpSecurityAttributes, uint 
dwCreationDisposition, uint dwFlagsAndAttributes, uint hTemplateFile);

private const int MY_CODE_PAGE = 437;
private const uint GENERIC_WRITE = 0x40000000;
private const uint FILE_SHARE_WRITE = 0x2;        
private const uint OPEN_EXISTING = 0x3;

i have taken this code from the given link.

You should include a description of the answer here in case the content of the link changes. — , Aug 16 '19 at 00:41

No console output when using AllocConsole and target architecture x86

5 Answers5

Linked

Related