Data Execution Prevention with Windows Services

Question

I am running Windows 7 Ultimate 64 bit.

I have a windows service (written in C#) that calls into a dll released by an major telecoms service provider here in South Africa (TELKOM). The dll is called MPIEst.dll, and I believe it was written in C++. The guy at the link (http://social.msdn.microsoft.com/Forums/en-US/windowscompatibility/thread/a7e5aafc-bb52-42c3-a3b7-19cb4cfbf6d5/) sort of had the same problem I had.

After doing some research I found out that DEP (Data Execution Prevention) was to blame for the error "Unable to load DLL 'MPIest.dll': Invalid access to memory location. (Exception from HRESULT: 0x800703E6)". So I thought "ok that's easy...let's turn off DEP for the whole computer and see what happens". So I do this and the result becomes even weirder...The windows service runs successfully like nothing happens, but all calls into the DLL don't do anything at all.

I know this because when I wrote a normal C# console app that uses the DLL, the DLL methods returned values and worked properly, but when run from within the windows service, the DLL methods return a non-zero result, which indicates that something went wrong. The thing is there is no documentation that says what the return codes mean.

Anyway the bottom line is that somehow turning off DEP for the whole system doesn't affect a windows service. Has anyone encountered this before? The windows service and the console app call exactly the same code and do exactly the same thing, but the console app works properly, whilst the windows services silently doesn't do anything but DEP is turned off for the whole system.

Thanks in advance.

Answer 1

Found the solution. The solution is actually two-fold. First there is DEP, which you need to turn off for the whole computer, and then restart the computer (this is because one cannot turn off DEP for a windows service via control panel).

Then secondly, I used the "ProcessMonitor" application to see what the windows service and the "MPIEst.dll" were doing with the windows OS behind the scenes. It turns out that the DLL was searching for a file it depended on (client.mpi) in the system directory folder, even though the file was in the same directory as the windows service executable. So I added code to copy the necessary files to the system directory and everything worked.