-2

I would like to know if there is a Win32 function to stop the console from opening. It already does this for a int WINAPI wWinMain entry point function but with the regular int main entry point function, the console opens by itself. I want to know if Windows has a function to close it. I am using llvm clang.

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
Evan
  • 31
  • 3
  • Where should the output be written instead? Why are you writing to stdout if you don't want it to be seen? – Barmar Apr 27 '22 at 03:37
  • @Barmar I can change where the output is going, that's fine. The problem is that the console window is still there – Evan Apr 27 '22 at 03:38
  • if you don't want a console then create a WinForm app instead and hide the form – phuclv Apr 27 '22 at 03:40
  • @phuclv I'm writing a game engine. I need the ugly std out window to be gone in release mode. I need the output for debugging but it's tacky in a real app – Evan Apr 27 '22 at 03:42
  • 3
    If you are using Microsoft Visual Studio, you can use the [`/SUBSYSTEM`](https://learn.microsoft.com/en-us/cpp/build/reference/subsystem-specify-subsystem?view=msvc-170) linker option. With `/SUBSYSTEM:CONSOLE`, you will automatically get a console window that is linked to standard input and standard output. With `/SUBSYSTEM:WINDOWS`, this will not happen, but you can still set up a console manually by using the [Console API](https://learn.microsoft.com/en-us/windows/console/consoles). – Andreas Wenzel Apr 27 '22 at 03:47
  • 1
    @Evan a GUI (`wWinMain()`) app can create a console window if it wants one. Look at [`AllocConsole()`](https://learn.microsoft.com/en-us/windows/console/allocconsole). – Remy Lebeau Apr 27 '22 at 03:51
  • @RemyLebeau I know that a wWinMain app can have a console. I don't see how that has anything to do with the question – Evan Apr 27 '22 at 03:55
  • 1
    @Evan you would design your engine to run exclusively as a GUI app (even if it has no GUI) so that there is no console created by default, and then in your debug mode only, you can allocate a console window to write your debug output to. – Remy Lebeau Apr 27 '22 at 04:12
  • 3
    Consider using a more appropriate system for reporting diagnostics, such as [Event Tracing for Windows (ETW)](https://learn.microsoft.com/en-us/windows-hardware/drivers/devtest/event-tracing-for-windows--etw-). – IInspectable Apr 27 '22 at 06:00
  • 1
    Why exactly can't you use WinMain though? – Lundin Apr 27 '22 at 12:55

2 Answers2

1

This might be controlled by the compiler.

For example, in the GCC documentation one can use the flag -mconsole if he wants a console application or the flag -mwindows if he wants a GUI application, without any console.

You probably have similar flags for Microsoft Compiler, you might want to read the documentation.

Robert
  • 2,711
  • 7
  • 15
  • First off, thank you for having an actual answer. Second, I'm using llvm clang on Windows. Do you know of an option for that? I'm fine digging through documentation if no one has an answer – Evan Apr 27 '22 at 03:44
  • @Evan: This may answer your follow-up question: [Clang's equivalent to GCC's -mwindows](https://stackoverflow.com/q/35057242/12149471) – Andreas Wenzel Apr 27 '22 at 03:54
  • this is the same as `/SUBSYSTEM` option in MSVC as commented above – phuclv Apr 27 '22 at 13:16
  • It has nothing to do with the compiler. It is the behavior of the operating system to automatically allocate a console for an application targeting the console subsystem. The linker sets which subsystem the application targets by writing a particular field in the PE. So, yes, if you don't want a console allocated, then don't tell the linker that you're building a console application. – Cody Gray - on strike Apr 28 '22 at 03:43
0

I found the solution. The linker command -Wl,/subsystem:windows,/entry:mainCRTStartup will tell it that it is a GUI app but to use int main as the entry point

Evan
  • 31
  • 3
  • Forcibly changing the entrypoint like this is a rather dangerous thing to do. The initialization that the operating system loader does for a GUI application is very different from the initialization that it does for a console application. By combining the switches in this way, you've created a weird sort of hybrid that is obviously not a tested or supported configuration. Note that the *actual* entry point for a PE binary on Windows is `ULONG __stdcall function(void* Peb)`. This underlying entry point will call either `[w]mainCRTStartup` for an app targeting the console subsystem, ... – Cody Gray - on strike Apr 28 '22 at 03:50
  • …or `[w]WinMainCRTStartup` for an app targeting the Windows subsystem. The `[w]main` or `[w]WinMain` functions provided by your code are called by the previously enumerated functions, respectively. That's why this appears to work, but it is not a reasonable thing to do, as you've told the linker to create an app that targets the Windows subsystem but starts up like a console app. If you want to have a console window but not create it by default, just set the Windows subsystem then call `AllocConsole` when you want/need the console. – Cody Gray - on strike Apr 28 '22 at 03:52
  • There is no reason why you should want to use `int main` as the entry point for an application that runs on Windows. If it's for interoperability with a cross-platform code base, conditionally define a `[w]WinMain` function, and have it call your cross-platform `main` function. – Cody Gray - on strike Apr 28 '22 at 03:56