0

I'm playing with a TEC-Microsystem device (DX5100), it provides a C++ dll with an API to open connection:

HANDLE OpenSerialPort(char *PortName, DWORD SupporedBaud);

For debugging pruprose, I'd like to trace every data being sent/received to the port (a kind of "sniffer" working within the same process). As I have access to the Windows API handle (HANDLE returned value), is there a way to setup any "listener" using Win API to know when data is being sent to/received from the USB port?

jpo38
  • 20,821
  • 10
  • 70
  • 151
  • 2
    If you're using a third-party API there's no guarantee that this handle might represent the serial port itself, it could be a handle to some API-internal structure. – Some programmer dude Jun 04 '20 at 14:15
  • Even if the HANDLE is actually a Win32 handle (as it is likely to be judging from the function declaration), there is no way to monitor it's i/o at the user level. – Michael Chourdakis Jun 04 '20 at 14:32
  • @Someprogrammerdude: Sure, that's definitely true, but if by chance this is the result of a win32 `CreateFile` function, I'd like to give this a try... – jpo38 Jun 04 '20 at 14:37
  • @MichaelChourdakis: That's what I fear, but as some sniffer software (like "Device Monitoring Studio") is able to do it, I was wondering if a C++ program could do it too...which appears to be even easier if you have a handle to the connection. – jpo38 Jun 04 '20 at 14:38
  • @jpo38 you 'd need a kernel mode driver. Or, at the user level, a hook of ReadFile/WriteFile perhaps. – Michael Chourdakis Jun 04 '20 at 14:39
  • @MichaelChourdakis: How can a hook of ReadFile/WriteFile be setup, do you have an example? – jpo38 Jun 04 '20 at 14:47
  • @jpo38 you 'd need to patch a process import table to redirect the calls from ReadFile/WriteFile to your functions. See API hooking libraries, for example [mhook](http://codefromthe70s.org/mhook24.aspx) – Michael Chourdakis Jun 04 '20 at 14:48
  • @MichaelChourdakis: Looks less trivial than expected ;-). Looks like Detours could also help (https://stackoverflow.com/a/873659/3336423). You may want to post all this as an answer. – jpo38 Jun 04 '20 at 14:53
  • @MichaelChourdakis: Can all this be done directly in the current process? – jpo38 Jun 04 '20 at 14:54

1 Answers1

1

As @Some programmer dude says, there is no guarantee that this is a valid Windows handle, but even if so, your options are limited:

  1. You would need a kernel mode driver to spy on i/o to the handle
  2. You would need to patch reading/writing functions to the handle using a library such as Detours or mhook.

The latter approach (as any API hooking technique at the user level) is unreliable, for if the target process uses any sort of trickery to access the handle you won't be notified.

It can be done within the same process by self-patching (changing the IAT table), or in a remote process with injection, either by the APP_Init DLL or some other injection technique like LoadLibrary/CreateRemoteThread which would do the patching. In any case you would have to forward the request to the actual ReadFile/WriteFile/DeviceIOControl function after you log it.

To change the IAT table, you might want to refer to my Load EXE as DLL article which uses the same techique in an unrelated mission. This article, this article and this article have also more information.

Michael Chourdakis
  • 10,345
  • 3
  • 42
  • 78