0

Related to the answer found here: https://stackoverflow.com/a/29572621/4055837

Using shell32.dll SHOpenWithDialog to prompt the user to select an app to open a file.

Is there a way for me to programmatically detect if the user has cancelled the dialog (either using the cancel button or the [x] to close the dialog? The result that comes back (0x0) is the same result I receive if the user selects an app, which does not allow me to take a different code path if the user changes their mind about opening a file with a specified app.

[edit for clarification] For the scenario I am working with, I need to be able to tell the difference between user cancelling "open with" and user selecting an app in order to direct my application's handling of information related to the file. That app may or may not already be in the registry's OpenWith key.

For files with unusual extensions and files with multiple or custom options, the open behavior may launch open with; for instance, a test file called foo.file that has the OpenWith key and an entry for notepad++.exe still fires the dialog to ask me if I still want to use notepad++.

For many file types my existing checks regarding file and parent folder locking work, but for some extensions/editors, there is no lock being maintained. In those cases, I can detect changes in the file by periodically checking for updates to the file, but that does not help when I need to respond to the user deciding to cancel the "open with" dialog.

[further clarification] The use case calls for supporting the ability for the customer to open a file that may or may not have an associated executable in an executable of their choosing. This is not information that is available until runtime. The application tracks files that should be secured; when an authorized user needs to open a file for editing or execution, the application needs to support the operation required on demand.

  • If they change their mind and cancel the Open With, then the file needs to be left in a secure state.
  • If they select the executable that they want to open the file with, then the application releases the file to that executable.
Sean
  • 56
  • 7
  • Read the current value from the registry, invoke the dialog, and then read the current value from the registry. Compare the two values you read. If they're the same, the user cancelled or made a meaningless change. – Ken White May 24 '17 at 22:22
  • That will help me detect if they make a new change, but for the scenario I'm working within, the information I need to capture is whether or not they opened the file at all, not just if they changed that data. For many applications that edit files, I can check for a lock on that file or its containing folder (and do) to see if the file is open. However, there are applications that will load the content into memory/temp file(s) that will not hold a lock on the original file/folder, which limits how reliable this approach is. – Sean May 24 '17 at 22:42
  • Then why are you invoking the *open with* dialog at all? Use ShellExecute with a NULL verb, which will use the default action of the default application, and return a value that lets you know if the command succeeded or not. – Ken White May 24 '17 at 22:48
  • My concern is that the application needs to be generic enough to handle customers using proprietary/unknown file formats and editors without losing the ability to respond to user actions on the files that the customer wants tracked by the application. – Sean May 24 '17 at 23:00
  • OK. If the customer has a proprietary file format that has an associated editor, ShellExecute will launch it. If they're using a standard associated app, ShellExecute will open it. If there's no association at all, ShellExecute will return a value to indicate that as well. If you have the filename, you can track it. I'm at a loss as to your requirements, and since your post does not describe them clearly in order to do so, I'm out. Good luck. – Ken White May 25 '17 at 00:11
  • Initial vagueness of specifications was because the application is a data security application. I don't have the liberty to divulge details beyond the minimum necessary (which is a moving target, unfortunately). – Sean May 25 '17 at 16:10

0 Answers0