2

I have run into an issue while working with numpy's savetxt function. I have been trying to save data deep inside a file structure. Below is a minimal working example (for you to run it you will need to make the following directories, however).

import numpy as np

dir1 = "LongDirectoryName/"
dir2 = "VeryLongDirectoryName/"
dir3 = "EvenLongerDirectoryName/"
dir4 = "LongestOfThemAllDirectoryName/"

filename = "../" + dir1 + dir2 + dir3 + dir4 + "longfilename_with_129_charss.txt"   # this works
#filename = "../" + dir1 + dir2 + dir3 + dir4 + "longfilename_with_130_charsss.txt" # this does not 
print(len(filename))
myarray = np.array([1,2,3])
np.savetxt(filename, myarray)

The relative filepath is not an issue because the 129 character filename does work. It seems to me that 129 characters is the limit. When I go to try the 130 character filename, I receive the following error:

File "C:\Users\njkro\Anaconda3\lib\site-packages\numpy\lib\npyio.py", line 1359, in savetxt
    open(fname, 'wt').close()
FileNotFoundError: [Errno 2] No such file or directory: '../LongDirectoryName/VeryLongDirectoryName/EvenLongerDirectoryName/LongestOfThemAllDirectoryName/longfilename_with_130_charsss.txt'

Other Information
I am using Windows 10.
I am using Anaconda, and the version of numpy I am using is 1.16.2.

My questions:

  1. Can anyone confirm this?
  2. Is there an explanation for this character limit? I would understand a 128 character limit more than 129.
  3. Is there a workaround (for long filenames) if I can't change the file system?
Nathaniel Kroeger
  • 121
  • 4
  • 1
    Works fine when I try it (and yes, I did use the commented-out version, not the "this works" version). – user2357112 Jun 27 '20 at 05:04
  • 1
    Is just the filename 130 char, or is it the whole path? The issue might be the length of the entire filepath being greater than Windows' MAX_PATH. https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file#:~:text=In%20the%20Windows%20API%20(with,and%20a%20terminating%20null%20character. – captnsupremo Jun 27 '20 at 05:31
  • 1
    I answered; would you mind changing the title of your question to something like: "FileNotFoundError on long pathname in Python in Windows"? The error occurs at the `open()` function, so it doesn't have to do with numpy. – Han-Kwang Nienhuys Jun 27 '20 at 12:30

1 Answers1

4

It appears that Windows produces an error based on the absolute path. Even if you try to get into a deeply nested directory by chdir'ing in steps, it will fail if the absolute path is too long.

You can still get there by resorting to the FAT32-style 8-character filename aliases (even when on NTFS), for example, rather than

open('abcdefghiabcdefghiabcdefghi/abcdefghiabcdefghiabcdefghi', 'w')

you do

open('ABCDEF~1/ABCDEF~1', 'w')

but it is rather involved to reliably get these short filenames.

Alternatively: change a registry setting to allow long path names on this particular Windows system (use regedit.exe): under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem, set LongPathsEnabled to DWORD value 1.

Han-Kwang Nienhuys
  • 3,084
  • 2
  • 12
  • 31
  • Man, you're a genius! Thanks a lot! I would have spent hours finding that `LongPathsEnabled` flag by myself. – xfra35 Dec 31 '22 at 09:07