1

A C library I use requires a FILE* opened to a COM port to work with it, but I have to set up the port (connection speed, etc) before passing it to a library. On Windows, this is done using SetCommState(HANDLE, DCB*).

Transformation between HANDLE and FILE* is a solved problem (How make FILE* from HANDLE in WinApi? and How do I get the file HANDLE from the fopen FILE structure?), but, surprisingly, it fails in different ways when applied to COM ports obtained via Bluetooth (RFCOMM/SPP).

If I first open a HANDLE and set COM port parameters, _open_osfhandle always fails for me:

HANDLE qc9200_handle = CreateFile(T("\\\\.\\COM9"), GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
/* this returns a valid value like 0x30 */
SetCommState(qc9200_handle, &qc9200_dcb); /* also succeeds */
int qc9200_fd = _open_osfhandle((intptr_t)qc9200_handle, _O_RDWR|_O_BINARY);
/* returns -1 and sets errno=EINVAL (22) */

I tried setting various combinations of flags from _O_APPEND (the only one which is both mentioned in the docs and makes sense with my application) to _O_RDWR|_O_BINARY (which I initially thought should have worked), but the errno is always EINVAL.

If I try to start with a FILE* or int file_descriptor to produce a HANDLE from it later, both _topen(path, _O_RDWR|_O_BINARY) and _tfopen(path, "r+b") fail right away and set errno=EACCES (13) no matter what access mode I request, even when I launch my application with Administrator rights. They don't fail immediately, though, it takes about a second, and I can see the light flashing on my USB dongle before the failure. Trying to open(COMPORT, "+<", "\\\\.\\COM9") the port from Perl also fails with "access denied".

Fishing for events with ProcessMonitor, surprisingly, only gives a bunch of RegQueryValue to HKLM\System\CurrentControlSet\Enum\BTHENUM_LOCALMFG\Device parameters\{Port Name, Authenticated, Encrypted} and one IRP_MJ_READ to %WINDIR%\system32\ucrtbased.dll while fopen is on the stack.

The COM port in question is obtained using ordinary Windows 7 Bluetooth settings, and every terminal program capable of working with COM ports does access it, too, even if not run as administrator. When I do same things to a virtual COM port (com0com), everything works as expected; the errors persist only when I'm using a Bluetooth COM port.

UPD: As GetFileType(HANDLE) reveals, the handle to a Bluetooth COM port is not a file-HANDLE, nor it is anything GetFileType knows about. That's why C runtime library refuses to return a file descriptor to the handle and that's probably why fopen refuses to open the port. I'll have to implement an fprintf-like function which accepts HANDLEs instead and #ifdef and use it on Windows.

Community
  • 1
  • 1
aitap
  • 325
  • 1
  • 9
  • Try messing with the flags of `_open_osfhandle`. Perhaps removing `_O_RDWR` and/or adding `_O_APPEND` will make a difference. There is some code in [legacy_ntp](https://github.com/dan-boa/legacy_ntp/blob/master/ports/winnt/ntpd/win32_io.c) which uses `_open_osfhandle` on serial ports, although it uses `_O_TEXT` rather than `_O_BINARY`. Once you have a valid file descriptor, you can use `_fdopen` to open a `FILE*` (but you knew that already). – Ian Abbott Feb 05 '16 at 17:19
  • @Ian, No matter what flags of `_open_osfhandle` I try, it's always -1 and EINVAL. It's like `_open_osfhandle` doesn't like my HANDLE. But why should `fopen` on COM port fail when `CreateFile` succeeds - that's a true mystery. And why do `open`/`fopen` always return EACCES when I clearly have the access to the port using whatever terminal emulator? – aitap Feb 07 '16 at 18:21

1 Answers1

0

Thanks to a Russian language forum, I was able to solve this.

After opening a HANDLE to the virtual COM port, I called GetFileType() on it,

printf("handle=%d\nGetFileType=%lu\n", handle, GetFileType(handle));

Which resulted in

handle=36
GetFileType=0

with GetLastError returning 0.

GetFileType(HANDLE) == 0x0000 means FILE_TYPE_UNKNOWN.

It seems that CRT doesn't know how to properly produce FILE* from unknown-type HANDLES, so my only way to run a vfprintf-like function to write to the serial port would be to implement such a function for HANDLEs myself.

Given that it's a bad idea to use FILE* with serial ports because the standard library would issue seek() calls between reads and writes, and serial ports are unseekable, I had to drop FILE* completely and use plain int file descriptors on POSIX and HANDLEs on Windows.

aitap
  • 325
  • 1
  • 9