0

While I understand code that is not marked "executable" will trigger a DEP crash, I am trying to understand what type of common coding practices (in legacy Windows apps) would result in this type of crash.

Remy Lebeau
  • 555,201
  • 31
  • 458
  • 770
Malcolm McCaffery
  • 2,468
  • 1
  • 22
  • 43
  • On which operating system? – Basile Starynkevitch May 04 '20 at 04:37
  • 1
    "Common" coding practices? Hopefully none, as few "common" coding practices would include putting code in the data or stack segments for execution. The only place this is "common" would be in hacks, cracks, viruses or other similar things. – Some programmer dude May 04 '20 at 04:37
  • 1
    If you fill an unsigned character array with binary machine code instructions and then try to call it as a function. Something like what this guy was trying to do: https://stackoverflow.com/questions/55674951/how-to-run-machine-code-as-a-function-in-c or https://stackoverflow.com/questions/18476002/execute-binary-machine-code-from-c – Jerry Jeremiah May 04 '20 at 04:38
  • @Someprogrammerdude it's common problem in legacy Windows applications at least, including fairly mainstream applications... i debug/patch this problem all the time, but I never work from source code, so wonder how they were created in the first place – Malcolm McCaffery May 04 '20 at 05:00

1 Answers1

4

Something like this:


int main()
{
    char* s = (char*)malloc(1);
    s[0] = '\xC3';
    void (*p)() = (void (*)())(s);
    p();
}

ATL did this to allocate thunk for WndProc. The purpose of such WndProc thunks is to embed context parameter and use a method for WndProc instead of a function not taking extra context parameter.

The fix is easy enough, and does not necessarily include removal of dynamic code allocation:

  • one way is as @Remy pointed out allocate using VirtualAlloc and manage rights using VirtualProtect to make sure execution right is there.
  • easier way is to create a heap with HeapCreate and pass HEAP_CREATE_ENABLE_EXECUTE, allocate code on that heap
  • finally, there are ATL thunk helpers, stating from Windows 10, they can help avoiding having code generation implemented in the program or library. Though this will only work for thunks like ATL thunks, and not a generic solution.
Alex Guteniev
  • 12,039
  • 2
  • 34
  • 79
  • 2
    This is very easy to fix to support DEP, though. Use `VirtualAlloc()` instead of `malloc()`, and use `VirtualProtect()` to grant the memory execution rights after it is populated with machine instructions. – Remy Lebeau May 04 '20 at 07:25
  • 1
    I've elaborated possible fixes – Alex Guteniev May 04 '20 at 07:33