3

On my system the .xyz extension is not registered at all. So when I double click on an .xyz file in the Explorer, Windows 10 pops up the standard "How do you want to open this file" dialog which is expected.

Now when I run this short snippet, the same dialog as above is displayed, even though I put the SEE_MASK_FLAG_NO_UI flag in sei.fMask:

  SHELLEXECUTEINFO sei = { 0 };
  sei.cbSize = sizeof(SHELLEXECUTEINFO) ;
  sei.fMask = SEE_MASK_FLAG_NO_UI;
  sei.hwnd = AfxGetMainWnd()->GetSafeHwnd();
  sei.lpVerb = _T("open");
  sei.lpFile = _T("C:\\Users\\Test\\Documents\\temp\\Temp.xyz");
  sei.lpParameters = NULL;
  sei.lpDirectory = appdir;
  sei.nShow = SW_SHOW;
  ShellExecuteEx(&sei);

  • The file "C:\\Users\\Test\\Documents\\temp\\Temp.xyz" exists.

  • The same code with sei.lpFile pointing to an existing .txt opens it with Notepad as expected.

The documentation pretends that no error message should be displayed with the SEE_MASK_FLAG_NO_UI.

So what can I do so ShellExecuteEx does not pop up any message but returns an error code instead?

I only checked this on Windows 10, I don't know what the behaviour is on older Windows versions.

Jabberwocky
  • 48,281
  • 17
  • 65
  • 115
  • I guess that's not an error dialog is it. It's a dialog to allow the user to form an association so that the shell can then attempt to invoke the verb. – David Heffernan Mar 08 '19 at 13:48
  • @DavidHeffernan, exactly, it's not an error message. Anyway I don't want any UI displayed. Maybe I should simply check if the ".xyz" key exists in the registry under HKEY_CLASSES_ROOT, and if not, short circuit `ShellExecuteEx` alltogether. – Jabberwocky Mar 08 '19 at 14:06

1 Answers1

2

Use FindExecutable() or AssocQueryString() before calling ShellExecute(). This allows to check if an association exists. SEE_MASK_FLAG_NO_UI prevents error displaying, however this situation is not treated as an error.

Michael Chourdakis
  • 10,345
  • 3
  • 42
  • 78
  • 2
    This is not 100% reliable, even if those functions say no there could still be a shell extension .DLL that handles the default action when you execute the filetype. – Anders Mar 08 '19 at 15:48
  • Where did you find that? – Michael Chourdakis Mar 08 '19 at 15:50
  • 1
    Shell extension .DLLs can dynamically decide if they are the default action, there is no way to know without actually executing their code. – Anders Mar 08 '19 at 15:51
  • 1
    @Michael Anders is right. I've written a Shell Extension that does exactly that. It dynamically takes over the default action. If the file being executed has a certain header, my app is executed, otherwise the normal default app is executed instead. See [Registering a Shortcut Menu Handler with a Dynamic Verb](https://learn.microsoft.com/en-us/windows/desktop/shell/shortcut-menu-using-dynamic-verbs#registering-a-shortcut-menu-handler-with-a-dynamic-verb) regarding the use of the `MayChangeDefaultMenu` Registry setting in the Shell Extension's registration – Remy Lebeau Mar 08 '19 at 18:05
  • 1
    OK in that case you can't do anything. The functions I 've provided might return a false negative, but never a false positive so he will never get a nasty "open with" window. – Michael Chourdakis Mar 08 '19 at 18:08