0

I have the following very simple code (edited):

public void MyFunction(string inPath, string inArguments)
{
    // Configure the process using the StartInfo properties
    Process process                 = new Process();
    process.StartInfo.FileName      = inPath;
    process.StartInfo.Arguments     = inArguments;
    process.StartInfo.WindowStyle   = ProcessWindowStyle.Normal;
    process.Start();


    uint ENABLE_QUICK_EDIT = 0x0040;
    IntPtr handle;
    UInt32 consoleMode;


    // Attempt type 1
    // (get external process handle via window name - fails, QuickEdit mode still active)
    handle = FindWindowByCaption(IntPtr.Zero, "Window_Name");
    GetConsoleMode(handle, out consoleMode);  // fails
    consoleMode &= ~ENABLE_QUICK_EDIT;
    SetConsoleMode(handle, consoleMode);      // fails

    // Attempt type 2
    // (get external process handle via explicit process ID - fails, QuickEdit mode still active)
    AttachConsole((uint)process.Id);
    handle = GetConsoleWindow();              // fails
    GetConsoleMode(handle, out consoleMode);
    consoleMode &= ~ENABLE_QUICK_EDIT;
    SetConsoleMode(handle, consoleMode);      // fails

    process.WaitForExit();                    // Waits here for the process to exit.
}

With the relevant native functions being (edited):

[System.Runtime.InteropServices.DllImport("user32.dll", EntryPoint = "FindWindow", SetLastError = true)]
    static extern IntPtr FindWindowByCaption(IntPtr zeroOnly, string lpWindowName);

[System.Runtime.InteropServices.DllImport("kernel32.dll", SetLastError = true, ExactSpelling = true)]
    static extern bool FreeConsole();

[System.Runtime.InteropServices.DllImport("kernel32.dll", SetLastError = true)]
    static extern bool AttachConsole(uint dwProcessId);

[System.Runtime.InteropServices.DllImport("kernel32.dll")]
    static extern IntPtr GetConsoleWindow();

[System.Runtime.InteropServices.DllImport("kernel32.dll")]
    static extern bool GetConsoleMode(IntPtr hConsoleHandle, out uint lpMode);

[System.Runtime.InteropServices.DllImport("kernel32.dll")]
    static extern bool SetConsoleMode(IntPtr hConsoleHandle, uint dwMode);

This code opens a command console as a call to the external .exe file referenced in inPath. What do I need to add to ensure that the new console window instance is created with QuickEdit mode turned off? Do I need to somehow map this property onto the object before it instantiates the console window?

I have been asking myself this question for several days at this point. I have found some solutions on the net, e.g. these:

EDIT links:

But still, nothing has worked for my case so far. Any help would be very much appreciated.

Thank you all very much in advance!

supersausage007
  • 93
  • 1
  • 10
  • So process A starts process B, and you want process A to turn off QuickEdit mode on process B? It might not be possible, but to try you have to somehow make process A get process B's console window handle. If you can do that, then you *might* be able to have process A do it as shown in my answer to the second of the two links you posted. – Jim Mischel Oct 27 '17 at 03:04
  • If the process that starts the console app does not have a console, it could call [AttachConsole](https://learn.microsoft.com/en-us/windows/console/attachconsole), disable QuickEdit mode, and then call [FreeConsole](https://learn.microsoft.com/en-us/windows/console/freeconsole). If the starting process is a console application itself, I don't know what it could do. Perhaps free its own console, attach to the new one, disable QuickEdit, free the new console, and then re-attach to its own (which would create a new console). – Jim Mischel Oct 27 '17 at 03:16
  • @Jim Mischel, Thank you for your comments! You are correct: Main Program A starts an executable B, and monitors that B is indeed running. B does have its own console, but I do not have access to its source code for editing to have it call AttachConsole... – supersausage007 Oct 27 '17 at 13:27
  • I wasn't clear. Have program A attach to program B's console. All it needs is to get program B's process id, and then call `AttachConsole`. And you already have B's process id, because you started the process. – Jim Mischel Oct 27 '17 at 14:15
  • @Jim Mischel, please have a look at my code. After retrieving the external process's handle either by window name or console title (both work), the call to _GetConsoleMode_ fails; so does the call to _SetConsoleMode_. Consequently the changes to ~ENABLE_QUICK_EDIT are not set. What I am doing wrong? – supersausage007 Nov 23 '17 at 01:21
  • The first attempt fails at least in part because you're passing the wrong window handle to `GetConsoleMode`. You need to get the console window handle that you obtain from `GetConsoleWindowHandle`. Most likely, you'll have to attach to the console before calling `GetConsoleMode`. In the second attempt, you need to check the return value of `AttachConsole`. Don't forget to check the last error if that fails (i.e. `SetLastError` in your `DllImpor`t, and `Marshal.GetLastWin32Error`). – Jim Mischel Nov 23 '17 at 04:42

0 Answers0