18

I'm trying to make a small program that could intercept the open process of a file.

The purpose is when an user double-click on a file in a given folder, windows would inform to the software, then it process that petition and return windows the data of the file.

Maybe there would be another solution like monitoring Open messages and force Windows to wait while the program prepare the contents of the file.

One application of this concept, could be to manage desencryption of a file in a transparent way to the user. In this context, the encrypted file would be on the disk and when the user open it ( with double-click on it or with some application such as notepad ), the background process would intercept that open event, desencrypt the file and give the contents of that file to the asking application.

It's a little bit strange concept, it could be like "Man In The Middle" network concept, but with files instead of network packets.

Thanks for reading.

HyLian
  • 4,999
  • 5
  • 33
  • 40

4 Answers4

15

The best way to do it to cover all cases of opening from any program would be via a file system filter driver. This may be too complex for your needs though.

Undo
  • 25,519
  • 37
  • 106
  • 129
Brian R. Bondy
  • 339,232
  • 124
  • 596
  • 636
14

You can use the trick that Process Explorer uses to replace itself with task manager. Basically create a key like this:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\taskmgr.exe

Where you replace 'taskmgr.exe' with the name of the process to intercept. Then add a string value called 'Debugger' that has the path to your executable. E.g:

Debugger -> "C:\windows\system32\notepad.exe"

Every a process is run that matches the image name your process will actually be called as a debugger for that process with the path to the actual process as an argument.

Sandeep Datta
  • 28,607
  • 15
  • 70
  • 90
Luke Quinane
  • 16,447
  • 13
  • 69
  • 88
  • I believe this would be the best way, but how could I use it for openfiledialogs? Open file dialogs are in comdlg32.dll right? – Vinicius Gonçalves Jun 30 '17 at 14:27
  • This works great, but note the potential infinite recursion in the case where your interceptor then wants to execute the original process. Easy workaround: https://stackoverflow.com/questions/1109564/intercept-windows-open-file – Ohad Schneider Oct 05 '20 at 15:52
9

You could use code injection and API redirection. You'd start your target process and then inject a DLL which hooks the windows API functions that you want to intercept. You then get called when the target process thinks it's calling OpenFile() or whatever and you can do what you like before passing the call on to the real API.

Google for "IAT hooking".

Len Holgate
  • 21,282
  • 4
  • 45
  • 92
0

Windows has an option to encrypt files on the disk (file->properties->advanced->encrypt) and this option is completely transparent to the applications.

Maybe to encrypt decrypt file portions of a disk you should consider softwares like criptainer?

There is this software as well http://www.truecrypt.org/downloads (free and open source) but I haven't tried it.

Developing a custom solution sounds very difficult.

mic.sca
  • 1,688
  • 2
  • 18
  • 34
  • 1
    Ok, you maybe can do it as a file system filter driver but I don't think it's a "viable" solution ..unless you have plenty of time and skills :-) and are developing something very specific that requires just that. – mic.sca Jul 10 '09 at 13:50
  • The crypt example was only to illustratre the doubt. I dont need any encryption at all :) – HyLian Jul 12 '09 at 21:15