1

I am trying to get a handle on Windows stdout handles.

I need to understand why there are different handles for STDOUT (CONOUT$?) and how to interpret these differences. I know there are different output buffers used by the Windows API, but failt to understand (from the many MSDN docs) when and how to, use them.

Using ctypes and python3, I can get 2 of these by the script below. However, running the script in (Cygwin) mintty/bash, PowerShell (6.1.1), CMD or even ConEmu, all yield slightly different results.

In mintty all the stdout handles are the same, while in PowerShell they are different. What is going on?

Question:
Why are the stdout handles obtained by: _get_osfhandle and GetStdHandle different?

Here is my code:

#!/usr/bin/env python3
#-*- coding: utf-8 -*-
#----------------------------------------------------------------------------
import sys
import ctypes
from ctypes import cdll, c_ulong

def color(text, color_code):
    return '\x1b[%sm%s\x1b[0m' % (color_code, text)
def byellow(text): return color(text, '1;49;33')
def printHex(mode):
    return byellow("0x%04x  (%s)" % (mode, mode))   # DWORD

kFile = 'C:\\Windows\\System32\\kernel32.dll'
mFile = 'C:\\Windows\\System32\\msvcrt.dll'

print("\n Getting Console STDOUT handles using 2 different methods:")

try: 
    k32    = cdll.LoadLibrary(kFile)
    msvcrt = cdll.LoadLibrary(mFile)
except OSError as e:
    print("ERROR: %s" % e)
    sys.exit(1)

try: 
    hmsvcrt_osf = msvcrt._get_osfhandle(sys.stdout.fileno())    # Get the parent (?) process Console Output buffer Handle
    hk32_11     = k32.GetStdHandle(-11)                         # Get the current process Console Output buffer Handle
except Exception as e:
    print("ERROR: %s" % e)
    sys.exit(1)

print(" Got stdout handles using:\n")
print("   msvcrt._get_osfhandle : %s" % printHex(hmsvcrt_osf))
print("   k32.GetStdHandle(-11) : %s" % printHex(hk32_11))

Here is the output:

# In PowerShell
# python3.6m.exe .\testHandles.py

 Getting STDOUT handles using 2 different methods:
 Got stdout handles using:

   msvcrt._get_osfhandle : 0x001c  (28)
   k32.GetStdHandle(-11) : 0x014c  (332)

# In Cygwin
# ./testHandles.py

 Getting STDOUT handles using 2 different methods:
 Got stdout handles using:

   msvcrt._get_osfhandle : 0x0338  (824)
   k32.GetStdHandle(-11) : 0x0338  (824)

# pwsh.exe -NoProfile -c "python3.6m.exe C:\test\testHandles.py"

 Getting STDOUT handles using 2 different methods:
 Got stdout handles using:

   msvcrt._get_osfhandle : 0x0338  (824)
   k32.GetStdHandle(-11) : 0x0144  (324)

According to MSDN:

  • _get_osfhandle:

    Retrieves the operating-system file handle that is associated with the specified file descriptor.

  • GetStdHandle:

    Retrieves a handle to the specified standard device (standard input, standard output, or standard error).

Related Questions:


ADDENDUM: 2018-12-14

In: /usr/lib/python3.6/ctypes/__init__.py:

  • class CDLL(object):

An instance of this class represents a loaded dll/shared library, exporting functions using the standard C calling convention (named 'cdecl' on Windows).

The exported functions can be accessed as attributes, or by indexing with the function name. Examples:

<obj>.qsort -> callable object

<obj>['qsort'] -> callable object

Calling the functions releases the Python GIL during the call and reacquires it afterwards.

  • class PyDLL(CDLL):

This class represents the Python library itself. It allows accessing Python API functions. The GIL is not released, and Python exceptions are handled correctly.

Then if _os.name == "nt":

  • class WinDLL(CDLL):

This class represents a dll exporting functions using the Windows stdcall calling convention.


From the discussion comments:

stdcall:

The stdcall[4] calling convention is a variation on the Pascal calling convention in which the callee is responsible for cleaning up the stack, but the parameters are pushed onto the stack in right-to-left order, as in the _cdecl calling convention. Registers EAX, ECX, and EDX are designated for use within the function. Return values are stored in the EAX register.

stdcall is the standard calling convention for the Microsoft Win32 API and for Open Watcom C++.

not2qubit
  • 14,531
  • 8
  • 95
  • 135
  • The standard handles aren't necessarily the same as the handles mapped to C file descriptors 0, 1, and 2 (could be duplicates or totally unrelated); and the standard I/O streams stdin, stdout, and stderr aren't necessarily file descriptors 0, 1, and 2. Also, the C runtime used by Python is unlikely to be msvcrt.dll -- unless you're using a special build such as Cygwin or MSYS (maybe, given the Unix-style name "python3.6m.exe"). To ensure it's the same, import `msvcrt`; don't load it via ctypes. – Eryk Sun Dec 14 '18 at 13:16
  • kernel32.dll should be loaded as `ctypes.WinDLL('kernel32', use_last_error=True)`. This ensures the proper stdcall calling convention for 32-bit processes. `use_last_error` captures the thread last error value immediately after a call to ensure it's protected. Access it via `ctypes.get_last_error()`. – Eryk Sun Dec 14 '18 at 13:20
  • As I described [here](https://stackoverflow.com/a/53776733/1147688), I can't use `WinDLL` (nor import `msvcrt`) since it doesn't exist in Cygwin's Python, AFAICT. What's the difference between `windll` and `cdll` anyway? (There's also a `pydll`.) – not2qubit Dec 14 '18 at 13:28
  • 1
    `WinDLL` uses the [stdcall calling convention](https://en.wikipedia.org/wiki/X86_calling_conventions#stdcall). There's no difference in a 64-bit Windows process. Cygwin has to use this convention internally when calling Windows functions, but Cygwin binaries only use `CDLL` (cdecl). I think the C runtime shared library is cygwin1.dll (check `ctypes.util.find_library('c')`), but it's not relevant to ask how Cygwin file descriptors map to Windows handles. Native handles are on the other side of the OS abstraction layer, and you don't have any use for them in POSIX programming. – Eryk Sun Dec 14 '18 at 18:04

0 Answers0