1

If we take a look to https://docs.python.org/3/library/pathlib.html#pathlib.Path.resolve source code we can see the routine is calling behind the curtains https://docs.python.org/3/library/os.path.html#os.path.realpath

I'm trying to really understand how os.path.realpath works in certain directories such as c:\windows\system32, ie:

>>> from pathlib import Path                                                    
>>> Path("c:/windows/system32")                                                 
WindowsPath('c:/windows/system32')                                              
>>> Path("c:/windows/system32").resolve()                                       
WindowsPath('C:/Windows/SysWOW64')                                              
>>> Path("c:/windows/system32").resolve(strict=True)                            
WindowsPath('C:/Windows/SysWOW64')                                              
>>> Path("c:/windows/system32").resolve()==Path("c:/windows/system32")          
False          

Or directories such as c:/windows/system32/openssh where you can get "unexpected results" such as below:

enter image description here

>>> list(Path("c:/windows/system32/openssh").resolve().glob("*"))
[]
>>> list(Path("c:/windows/system32/openssh").glob("*"))
[]

or

>>> os.listdir(r"C:\Windows\System32\OpenSSH")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Windows\\System32\\OpenSSH'

If you do dir /a you'll get

24/09/2022  10:03    <DIR>          System32

So you can see it's neither a SYMLINKD nor JUNCTION.

Could you explain how os.path.realpath works in these cases? Why can't i list/glob the content of that c:\windows\system32\openssh folder?

References: ntpath.py

Ulrich Eckhardt
  • 16,572
  • 3
  • 28
  • 55
BPL
  • 9,632
  • 9
  • 59
  • 117
  • I can't tell you why, but why is this a problem? Windows allows forward slashes everywhere – tripleee Sep 24 '22 at 10:38
  • @tripleee think the question the OP is asking is why does it resolve to `SysWOW64`... – Jon Clements Sep 24 '22 at 10:40
  • Here's the thing, I've realized about this "issue" when i was converting `os.getenv("PATH")` into pathlib Paths using resolve and I soon started to get unexpected results... Basically I want to know a reliable way to use my PATH directories to be used as search paths... I'm using that to collect dependencies recursively from a PE file. In short, I want to understand how os.path.realpath works in certain edge cases like the one showcased in the question. Exactly what @JonClements has clarified on his comment – BPL Sep 24 '22 at 10:46
  • Funny cos the cmdline utility `realpath` from portable git will tell me the real path of c:\windows\system32 is c:\windows\system32 , who do I trust here? :) – BPL Sep 24 '22 at 10:50
  • @tripleee No, Windows does not allow forward slashes everywhere. It's literally only a few distinct places where the Windows API permits using forward slashes as path delimiters, and only under certain conditions. The path separator on Windows is (and always has been) a backslash. – IInspectable Sep 24 '22 at 11:20
  • Anyway, you'll find the answer to the question documented here: [File System Redirector](https://learn.microsoft.com/en-us/windows/win32/winprog64/file-system-redirector). WoW64 is an emulator, not just a set of symbolic links (and it doesn't use symbolic links for redirection). – IInspectable Sep 24 '22 at 11:22
  • @IInspectable Nice, that link certainly helps, i was debugging a little bit here and I've found out the place where the resolution of the path is happening, which is [here](https://github.com/python/cpython/blob/de33df27aaf930be6a34027c530a651f0b4c91f5/Lib/ntpath.py#L591), now I need to find where this routine is living in cpython though – BPL Sep 24 '22 at 11:59
  • https://stackoverflow.com/a/57558880/3809375 – BPL Sep 24 '22 at 12:07
  • I think the answer to my original question is buried [here](https://github.com/python/cpython/blob/main/Modules/posixmodule.c#L4324-L4391) – BPL Sep 24 '22 at 12:10
  • At this point makes total sense how os.path.realpath works behind the curtains yet one question still remains... How to properly & reliably glob paths such as `c:/windows/system32/openssh`? :/ – BPL Sep 24 '22 at 13:22
  • "Globbing" and "reliable" are orthogonal. You can have either, but not both. – IInspectable Sep 24 '22 at 14:29

1 Answers1

-1

On windows, os.path.realpath is using GetFinalPathNameByHandleW behind the curtains, for more info you can check here.

If we make a simple experiment to see how that routine works:

#include <windows.h>
#include <iostream>

#define MAXPATHLEN 1024

void os__getfinalpathname_impl(const std::wstring &path) {
    std::wcout << L"analizing: " << path << std::endl;

    HANDLE hFile;
    wchar_t buf[MAXPATHLEN];
    int result_length;

    hFile = CreateFileW(path.c_str(), 0, 0, NULL, OPEN_EXISTING,
                        FILE_FLAG_BACKUP_SEMANTICS, NULL);

    if (hFile == INVALID_HANDLE_VALUE) {
        std::wcout << L"CreateFileW:" << path << std::endl;
    }

    result_length =
        GetFinalPathNameByHandleW(hFile, buf, MAXPATHLEN, VOLUME_NAME_DOS);

    if (!result_length) {
        std::wcout << L"GetFinalPathNameByHandleW" << std::endl;
    }

    std::wcout << L"result: " << buf << std::endl;
}

void main(int argc, char *argv[]) {
    std::wcout << L"test1" << std::endl;
    os__getfinalpathname_impl(L"c:/windows/system32");
    std::wcout << L"test2" << std::endl;
    os__getfinalpathname_impl(L"c:\\windows\\system32");
}

If we compile it for a 32-bit target we'll get:

test1
analizing: c:/windows/system32
result: \\?\C:\Windows\SysWOW64
test2
analizing: c:\windows\system32
result: \\?\C:\Windows\SysWOW64

If we compile it for a 64-bit target we'll get:

test1
analizing: c:/windows/system32
result: \\?\C:\Windows\System32
test2
analizing: c:\windows\system32
result: \\?\C:\Windows\System32

Now, if we want to confirm the above and checking the differences in a python context, we can do that easily by just spawning a couple oneliners:

python\3.10.6-amd64\python.exe -c "from pathlib import Path; print(Path('c:/windows/system32').resolve())"
C:\Windows\System32

python\3.10.1\python.exe -c "from pathlib import Path; print(Path('c:/windows/system32').resolve())"
C:\Windows\SysWOW64

Which makes total sense accordingly what's described in the below link https://learn.microsoft.com/en-us/windows/win32/winprog64/file-system-redirector

BPL
  • 9,632
  • 9
  • 59
  • 117
  • This does nothing to answer the question that was being asked. (Hint: You'll get pretty close if you compile your test code for a 64-bit target in addition to the 32-bit target you've been testing). – IInspectable Sep 24 '22 at 15:35
  • Nice! that's interesting, edited the question – BPL Sep 24 '22 at 16:21