6

The issue :

I have build https://github.com/reorg/pg_repack project whose generate a binary. This binary is linked with postgres 9.6 redistributable (I use those delievered by EntrepriseDb). All works fine on windows 7. I have no issue during the build or during the runtime. But on windows 8 or 10, the application always crashs with the following sequences. The binary is generated from C sources with visual studio 2013, on windows 7 (I have tried with a version generated on windows 10 but it doesn't change anythings), on a x64 system, for a x64 application and the optimisation is disabled and it use dynamic base. To be sure that I use the right binaries, I have copy all the redistributable binaries into the folder of my application.

Assembly details on windows 7 (working case) :

Few lines after the main, the application calls the function set_pglocale_pgservice

set_pglocale_pgservice(argv[0], "pgscripts");
00007FF6E4B39C85  mov         eax,8  
00007FF6E4B39C8A  imul        rax,rax,0  
00007FF6E4B39C8E  lea         rdx,[default_options+120h (07FF6E4B43B10h)]  
00007FF6E4B39C95  mov         rcx,qword ptr [argv]  
00007FF6E4B39C9A  mov         rcx,qword ptr [rcx+rax]  
00007FF6E4B39C9E  call        qword ptr [__imp_set_pglocale_pgservice (07FF6E4B40520h)]

Memory at 07FF6E4B40520h

0x00007FF6E4B40520  00000001403e1da0   .>@....
0x00007FF6E4B40528  0000000000000000  ........
0x00007FF6E4B40530  0000000000000000  ........
0x00007FF6E4B40538  00007ff6e4b35348  HS.äö...

(note : from 0x7FF6E4B40000 to 0x00007FF6E4B40560 memory contains functions addresses, and mapfile says : 0002:00000520 __imp_set_pglocale_pgservice postgres:postgres.exe [postgres redistributable, is linked dynamicly])

Then after the call qword ptr [__imp_set_pglocale_pgservice (07FF6E4B40520h)]

00000001403E1DA0  mov         qword ptr [rsp+18h],rbx  
00000001403E1DA5  push        rdi  
00000001403E1DA6  sub         rsp,0C40h  
00000001403E1DAD  mov         rax,qword ptr [1405F8C60h]  
00000001403E1DB4  xor         rax,rsp  
00000001403E1DB7  mov         qword ptr [rsp+0C30h],rax  
00000001403E1DBF  mov         rbx,rdx  
00000001403E1DC2  mov         rdi,rcx  
00000001403E1DC5  lea         rdx,[140430540h]  
00000001403E1DCC  mov         rcx,rbx  
00000001403E1DCF  call        00000001403F67FA

Then after the call 00000001403F67FA

00000001403F67FA  jmp         qword ptr [1403F8998h]

Memory at 1403F8998h

0x00000001403F8998  00007ffe87a5cc60  `Ì¥.þ...
0x00000001403F89A0  00007ffe87a47060  `p¤.þ...
0x00000001403F89A8  00007ffe87a5f8a4  ¤ø¥.þ...

(note : from 0x00000001403F8000 to 0x00000001403F8F08 memory contains functions address, and mapfile says : 0002:00000998 ??_C@_04FHBLDJDJ@?1bin?$AA@ libpgport:path.obj [postgres redistributable, is linked statically])

Then after the jump to 00007ffe87a5cc60

00007FFE87A5CC60  sub         rdx,rcx  
00007FFE87A5CC63  test        cl,7  
00007FFE87A5CC66  je          00007FFE87A5CC7C

... and all works fine

Assembly details on windows 10 (not working case) :

Few line after the main, the application calls the funciton set_pglocale_pgservice

set_pglocale_pgservice(argv[0], "pgscripts");
00007FF7E9879C85  mov         eax,8  
00007FF7E9879C8A  imul        rax,rax,0  
00007FF7E9879C8E  lea         rdx,[default_options+120h (07FF7E9883B10h)]  
00007FF7E9879C95  mov         rcx,qword ptr [argv]  
00007FF7E9879C9A  mov         rcx,qword ptr [rcx+rax]  
00007FF7E9879C9E  call        qword ptr [__imp_set_pglocale_pgservice (07FF7E9880520h)]

Memory at 07FF7E9880520h

0x00007FF7E9880520  00000001403e1da0   .>@....
0x00007FF7E9880528  0000000000000000  ........
0x00007FF7E9880530  0000000000000000  ........
0x00007FF7E9880538  00007ff7e9875348  HS.é÷...

(note : from 0x00007FF7E9880000 to 0x00007FF7E9880560 memory contains functions address, and mapfile says : 0002:00000520 __imp_set_pglocale_pgservice postgres:postgres.exe [postgres redistributable, is linked dynamicly])

Then after the call qword ptr [__imp_set_pglocale_pgservice (07FF7E9880520h)]

00000001403E1DA0  mov         qword ptr [rsp+18h],rbx  
00000001403E1DA5  push        rdi  
00000001403E1DA6  sub         rsp,0C40h  
00000001403E1DAD  mov         rax,qword ptr [1405F8C60h]  
00000001403E1DB4  xor         rax,rsp  
00000001403E1DB7  mov         qword ptr [rsp+0C30h],rax  
00000001403E1DBF  mov         rbx,rdx  
00000001403E1DC2  mov         rdi,rcx  
00000001403E1DC5  lea         rdx,[140430540h]  
00000001403E1DCC  mov         rcx,rbx  
00000001403E1DCF  call        00000001403F67FA

Then after the call 00000001403F67FA

00000001403F67FA  jmp         qword ptr [1403F8998h] (should call C@_04FHBLDJDJ@?1bin?$AA@ libpgport:path.obj [postgres redistributable, which is linked statically]))

Memory at 1403F8998h (here the application diverges from windows 7)

0x00000001403F8998  000000000059e6a2  ¢æY.....
0x00000001403F89A0  000000000059e6ac  ¾Y.....
0x00000001403F89A8  000000000059e6b6  ¶æY.....

(note : 0x00000001403F8998 is the address of op code in the middle of the function, not the address of a function)

00000001403F8998  mov         byte ptr [AC000000000059E6h],al  
00000001403F89A1  out         59h,al  
00000001403F89A3  add         byte ptr [rax],al  
...

Memory at 000000000059e6a2h

000000000059E69F  ?? ?? 
000000000059E6A0  ?? ?? 
000000000059E6A1  ?? ??

Then after the jump to 000000000059e6a2 => crash

Process monitor details on windows 7 for the loading of libraries (here libpq.dll):

[...]
"16:48:40,2946466","pg_repack.exe","7216","Load Image","C:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Image Base: 0x180000000, Image Size: 0x30000"
[...]

Process monitor details on windows 10 (here libpq.dll) (all is very similar to windows 7, except for the loading of libraries)

[...]
"11:52:20,6264717","pg_repack.exe","12464","QueryOpen","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","CreationTime: 21/08/2017 11:38:04, LastAccessTime: 21/08/2017 12:06:56, LastWriteTime: 09/05/2017 06:45:07, ChangeTime: 21/08/2017 18:04:09, AllocationSize: 184 320, EndOfFile: 183 296, FileAttributes: A"
"11:52:20,6265789","pg_repack.exe","12464","CreateFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Desired Access: Read Data/List Directory, Execute/Traverse, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Delete, AllocationSize: n/a, OpenResult: Opened"
"11:52:20,6266332","pg_repack.exe","12464","QuerySecurityFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Information: 0x20"
"11:52:20,6266513","pg_repack.exe","12464","CreateFileMapping","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","FILE LOCKED WITH ONLY READERS","SyncType: SyncTypeCreateSection, PageProtection: PAGE_EXECUTE"
"11:52:20,6266921","pg_repack.exe","12464","CreateFileMapping","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","SyncType: SyncTypeOther"
"11:52:20,6267619","pg_repack.exe","12464","Load Image","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Image Base: 0x180000000, Image Size: 0x30000"
"11:52:20,6274889","pg_repack.exe","12464","CreateFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Delete, AllocationSize: n/a, OpenResult: Opened"
"11:52:20,6275293","pg_repack.exe","12464","QuerySecurityFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Information: 0x20"
"11:52:20,6275471","pg_repack.exe","12464","QueryBasicInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","CreationTime: 21/08/2017 11:38:04, LastAccessTime: 21/08/2017 12:06:56, LastWriteTime: 09/05/2017 06:45:07, ChangeTime: 21/08/2017 18:04:09, FileAttributes: A"
"11:52:20,6276255","pg_repack.exe","12464","CloseFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS",""
"11:52:20,6291170","pg_repack.exe","12464","CloseFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS",""
[...]
"11:52:20,6539022","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","BUFFER OVERFLOW","Name: \testF"
"11:52:20,6539202","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","BUFFER OVERFLOW","Name: \testF"
"11:52:20,6539363","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","BUFFER OVERFLOW","Name: \testF"
"11:52:20,6539512","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","BUFFER OVERFLOW","Name: \testF"
"11:52:20,6539664","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","BUFFER OVERFLOW","Name: \testF"
"11:52:20,6603867","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Name: \testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll"
"11:52:20,6604319","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Name: \testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll"
"11:52:20,6604778","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Name: \testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll"
"11:52:20,6605211","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Name: \testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll"
"11:52:20,6605635","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","SUCCESS","Name: \testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll"
[...]

Note : I was expecting something similar to windows 7 or at least something like : 

"11:52:20,6539022","pg_repack.exe","12464","QueryNameInformationFile","B:\testFolder\pg_repack-master\msvc\bin\x64\Debug\libpq.dll","BUFFER OVERFLOW","Length: 63".

So I have no clue why this application has a so stranges behaviors on windows 8 or 10. I will be greatfull if someone would have some explainations or ideas in order to fix crashes. Don't hesitate to ask some details if needed.

Edeen
  • 229
  • 2
  • 10
  • Did you check to have the same settings for DEP and ASLR in both OS versions? – rakwaht Aug 22 '17 at 14:22
  • maybe that's why you have a problem.. It seems that your statically linked lib is not where you expected, if you have no clue you should look into executable space protection or osmething like that – rakwaht Aug 22 '17 at 14:26
  • 1
    you crash when [call](https://doxygen.postgresql.org/common_2exec_8c_source.html#l00559) `strcmp` imported function. and code at `07FFE87A5CC60` - this is `strcmp` begin code. in win10 dump visible that imports ( `1403F8998` this is part of *IAT*) not resolved. the `59e6a2` this is `RVA` of `IMAGE_IMPORT_BY_NAME` for `strcmp`. why is import not resolved, but pe not fail to load - good question. if have binary files (full complect) can debug and check this – RbMm Aug 22 '17 at 14:50
  • more concrete `MSVCR120.dll` is fail to load. as result `strcmp` which `postres.exe` import from `MSVCR120.dll` fail on call. but why `postgres.exe` not fail to initialize in this case ? – RbMm Aug 22 '17 at 15:26
  • @RbMm I will check your lead (need to read documentation on the subject). You can find the binaries at https://drive.google.com/file/d/0B-PUkzFlVTb9dHhadVJzMmhyTE0/view. – Edeen Aug 22 '17 at 15:45
  • @rakwaht DEP is disabled and ASLR enabled on each OS (and for this executable). – Edeen Aug 22 '17 at 15:46
  • MSCVR120.dll depends on KERNEL32. And if I check with dependency walker on windows 10, KERNEL32 depends on many dlls which cannot be resolved like API-MS-WIN-CORE-RTLSUPPORT-L1-2-0.DLL (even if msvcr120 x64 is installed). At the opposite all dll is resolved on windows 7. But it stays weird. For issue like this, windows prompts explicite messagebox and if KERNEL32 doesn't work, nothing would work. I will investigate the matter. – Edeen Aug 22 '17 at 16:54
  • @Edeen - with kernel32 no any problems, otherwise your windows not boot at all. `API-MS-WIN-*` exist - simply dependency walker invalid handle this names. and problem absolute not related to DEP or ASLR - this is fail load `MSCVR120.dll` – RbMm Aug 22 '17 at 18:08
  • 1
    understand - you use `postgres.exe` as dll. win10 not resolve exe import, when it loaded as dll – RbMm Aug 22 '17 at 18:40

1 Answers1

5

i look your binary package and view that you exec pg_repack.exe which statically import from postgres.exe. and root of problem - is import from exe file. more exactly from PE file which have not flag IMAGE_FILE_DLL in IMAGE_FILE_HEADER.Characteristics , because formal extension (exe, dll, etc not play role). for short i will in next under exe mean PE file without IMAGE_FILE_DLL in Characteristics, but not formal file extension

at first load exe as dll is incorrect - when exe loaded this way - it entry point not called. and can not be called, because it not designed be called as callback, receiving DLL_PROCESS_* notification, and call ExitProcess at the end. when entry point of module is not called - in general it not initialized. assume you call some exported function from exe, but what if this exported function use some data, which normally initialized in exe entry point ? so conclusion - we can call exported functions from exe only from dll loaded in this exe process. and in every process must be only one exe (as executable code).

at second specific postgres.exe have no relocation's ( IMAGE_FILE_RELOCS_STRIPPED flag in IMAGE_FILE_HEADER.Characteristics) - as result this PE can be loaded only at hard-coded address. this not problem for exe, which always mapped first to process, when almost all address space is free. but this in general is problem when PE loaded as DLL - not first in process - the hard-code imagebase can be already busy. so conclusion - you can not safe use any exe without relocation's as DLL

however the root of crash on windows 10 - because windows 10 not resolve PE ("exe") import if this pe have no flag IMAGE_FILE_DLL. in other words it process this PE like LoadLibraryEx with flag DONT_RESOLVE_DLL_REFERENCES - does not load additional executable modules that are referenced by the specified module and nor resolve imports. as result this PE not initialized and will crash at first import function call (in your case this is strcmp).

and this is happens only on win10. on win 8.1 (build 9600) and win 7 import for exe files (loaded as dll) resolved. (you say that on your win 8.1 it crashed - may be you use more new build or some update ? or better check)

the simplest test for this behavior call from test exe MatchToken, function, which exported from Netsh.exe. code can be next:

#include <Netsh.h>
#pragma comment(linker, "/defaultlib:Netsh.lib")

MatchToken(L"*", L"*");// crash here on win 10

crash on win10 because MatchToken internally try call _wcsnicmp from msvcrt.dll, but in win10 this import not resolved. but on win8.1, win7, win xp - this code work well.

little more complex example:

if (HMODULE hmod = LoadLibraryW(L"wshelper.dll"))
{
    DWORD (WINAPI * InitHelperDll)(_In_ DWORD dwNetshVersion, PVOID pReserved);

    if (*(void**)&InitHelperDll = GetProcAddress(hmod, "InitHelperDll"))
    {
        InitHelperDll(1, 0);// crash here on win10 only
    }
    FreeLibrary(hmod);
}

here we load standard windows NetShell helper dll - "wshelper.dll" and call InitHelperDll callback function. internally InitHelperDll call the RegisterHelper function from Netsh.exe. but again, because Netsh.exe not initialized (import not resolved) when it loades as DLL in win 10 - it crashed inside RegisterHelper

RbMm
  • 31,280
  • 3
  • 35
  • 56
  • Thanks a lot, I have done some tests and your explanations seems perfectly correct. I would have never figured it by myself, it was very informative. But I'm a little annoyed that the project pg_repack need a references on postgres.exe. I solve the issue by generating a static library of postgres in order to link with pg_repack. It works well (on windows 7/8/10), but I don't know if thete is a easier solution. Build a static library for each version of postgres is a huge work (relativly to the original purpose of build the tiny project of pg_repack). – Edeen Aug 24 '17 at 07:25
  • @Edeen i not familiar with concrete program *postgres*, and what sense that this *exe* export some api, but only one correct usage of *exe* export - *exe* load some dlls and this dlls import functions from this *exe*. are *postgres.exe* alow this model ? – RbMm Aug 24 '17 at 10:03
  • what is really can be done - you can build own *DLL* instead *EXE*, exec *postgres.exe* in suspended state, and inject own *DLL* to process by insert *APC*. in this case your *DLL* entry point will be called just before *EXE* entry in first thread (if only some av not also insert apc and force it execute early). here you can change entry point to own *DLL* and then work exactly as you do now. you can static link with *postgres.exe* in this *dll* and call it export. however question - if entrty point of *postgres.exe* will be not called - are it will be full initialized and safe call any it api – RbMm Aug 24 '17 at 11:57
  • 1) Postgres allow postgres extensions (dll loaded by postgres) to use exe export. But pg_repack is an external tools (executable). It links with postgres.exe in order to re-use some of the exported functions. Postgres and pg_repack are unix projects, this is maybe the reason pg_repack use this model. 2) The entry point of pg_repack do the initialization of postgres objects. It must be fully initialized. – Edeen Aug 24 '17 at 15:06