Should I deal with files longer than MAX_PATH?

Question

Just had an interesting case.

My software reported back a failure caused by a path being longer than MAX_PATH.

The path was just a plain old document in My Documents, e.g.:

C:\Documents and Settings\Bill\Some Stupid FOlder Name\A really ridiculously long file thats really very very very..........very long.pdf

Total length 269 characters (MAX_PATH==260).

The user wasn't using a external hard drive or anything like that. This was a file on an Windows managed drive.

So my question is this. Should I care?

I'm not saying can I deal with the long paths, I'm asking should I. Yes I'm aware of the "\?\" unicode hack on some Win32 APIs, but it seems this hack is not without risk (as it's changing the behaviour of the way the APIs parse paths) and also isn't supported by all APIs .

So anyway, let me just state my position/assertions:

First presumably the only way the user was able to break this limit is if the app she used uses the special Unicode hack. It's a PDF file, so maybe the PDF tool she used uses this hack.
I tried to reproduce this (by using the unicode hack) and experimented. What I found was that although the file appears in Explorer, I can do nothing with it. I can't open it, I can't choose "Properties" (Windows 7). Other common apps can't open the file (e.g. IE, Firefox, Notepad). Explorer will also not let me create files/dirs which are too long - it just refuses. Ditto for command line tool cmd.exe.

So basically, one could look at it this way: a rouge tool has allowed the user to create a file which is not accessible by a lot of Windows (e.g. Explorer). I could take the view that I shouldn't have to deal with this.

(As an aside, this isn't an vote of approval for a short max path length: I think 260 chars is a joke, I'm just saying that if Windows shell and some APIs can't handle > 260 then why should I?).

So, is this a fair view? Should I say "Not my problem"?

UPDATE: Just had another user with the same problem. This time an mp3 file. Am I missing something? How can these users be creating files that violate the MAX_PATH rule?

It's easy to create a file that has a name that is too long: just rename one of it's ancestor directories to be long enough to tip the file over the brink. In your example: `rename "C:\Documents and Settings\Bill\New Folder" "Some Stupid FOlder Name"`. — Adrian Pronk, Feb 05 '13 at 03:25

score 12 · Answer 1 · edited Jun 30 '17 at 10:02

12

It's not a real problem. NTFS support filenames up to 32K (32,767 wide characters). You need only use correct API and correct syntax of filenames. The base rule is: the filename should start with '\\?\' (see http://msdn.microsoft.com/en-us/library/aa365247(v=VS.85).aspx) like \\?\C:\Temp. The same syntax you can use with UNC: \\?\UNC\Server\share\Path. Important to understand that you can use only a small subset of API function. For example look at MSDN description of functions

CreateFile
CreateDirectory 
MoveFile

and so on

you will find text like :

In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\?\" to the path. For more information, see Naming a File.

This functions you can safe use. If you have a file handle from CreateFile you can use all other functions used hFile (ReadFile, WriteFile etc.) without any restriction.

If you write a program like virus scanner or backup software or some good software running on a server you should write your program so, that all file operations support filenames up to 32K characters and not MAX_PATH characters.

edited Jun 30 '17 at 10:02

David Heffernan

601,492
42
1,072
1,490

answered May 13 '10 at 10:05

Oleg

220,925
34
403
798

4

Hi thanks for the comment. Yes, I realise these APIs exist (see my post). I don't think one can just start using this hack and expect everything to be smooth sailing. For example, the \\?\ hack removes some parsing. Also, it's not clear to me how the OUTPUT of APIs is affected. It's just not clear enough to me what the impact is of this hack. If I go down this route I'll have to do a lot of testing. Do you have experience of using these APIs? – John May 14 '10 at 08:26
7

@John. You misunderstand how the way with “\\?\” works. It is **NOT** a hack! It is well documented way to work mostly directly with Native Windows NT API (from ntdll.dll) like NtCreateFile http://msdn.microsoft.com/en-us/library/bb432380%28VS.85%29.aspx. If you use this feature you will receive **not less security, but only less comfort**. The most of Windows API come from the time of 16-bit Windows 3.0 and have MAX_PATH restriction. The Native Windows NT API (from ntdll.dll) used mostly for driver programming. So usage of “\\?\” is safe. I have more as 15 years experience of usage this. – Oleg May 14 '10 at 10:10
4

OK yes "hack" is probably the wrong term :) What I'm saying however, is that it is NOT just a stand in replacement. For example, relative paths won't work, if you use "." or ".." in a path it won't work. Some APIs will also change OUTPUT of you use this "feature" - (e.g. GetLongPathName() will return paths which include the "\\?\"). On the other hand it's good to hear that you've had 15 years experience using this API - that eases my mind a lot. – John May 14 '10 at 13:54
1

This is the correct answer. Take a simple case of moving a directory with all children to another directory that bumps up against the MAX_PATH limit. Should you just fail all the files that can't fit into C:\myreallylong\stupid\directoryname\thatiwant\torestore\myoldbackups\tothisannoying\reallylongpath? ;) – GalacticJello Oct 24 '13 at 02:53
2

Prefixing UNC paths as \\?\host\share does NOT work. You have to use \\?\UNC\host\share. – Ilya Jul 31 '14 at 17:26
@Ilya: Thank you for fixing the tipping error. I'm glad that one still read the old post. :-). [The link](http://msdn.microsoft.com/en-us/library/aa365247(v=VS.85).aspx) to MSDN documentation which I referenced 4 years ago still work. In contains many other helpful names (`\\.\CdRomX`, `\\.C:`, `\\.\PhysicalDiskX`, `\\?\GLOBALROOT` and so on). I wanted just point to the possibility to break `MAX_PATH` limit and to show that the usage is really easy. – Oleg Jul 31 '14 at 19:27

score 10 · Answer 2 · answered May 13 '10 at 11:57

This limitation is baked into a lot of software written in C or C++. Including MSFT code, although they've been chipping away at it. It is only partly a Win32 limitation, it still has a hard upper limit on the length of a file name (not path) through WIN32_FIND_DATA for example. One reason that even .NET has length restrictions. This is not going away any time soon, Win32 is still going strong and the stone-age C string won't disappear.

Your customer will have little sympathy with it, no doubt, probably until you can show them another program that fails the same way. Do however make sure that your code reliably can detect the potential string buffer overflow, followed by a reasonable diagnostic. No sympathy for programs bombing on heap corruption.

Hi Hans, thanks for response. You're helping me out on a couple of issues at the moment! Thanks! — John, May 14 '10 at 08:26

score 6 · Answer 3 · answered May 14 '10 at 02:13

As you mentioned many of the Windows Shell functions only work on paths up to MAX_PATH. Windows XP and I believe Vista both have problems in Explorer when nesting directories with long file names. I've not checked Windows 7 - perhaps they have fixed that. This unfortunately means that users have a hard time browsing these file.

If you really wish to support long paths you'll need to check any functions you are using in Shell32.dll that take paths to ensure they support long paths. For those that don't you'll have to use write them yourself using Kernel32 functions.

If you decide to use Shell32 and be limited to MAX_PATH, writing your code to support long file paths would be advisable. If Microsoft later change Shell32 (or create an alternative), you will be better positioned to add support for them.

Just to add another couple of dimensions to the problem, remember that filenames are UTF-16, and you may encounter non NTFS or FAT filesystems that may be case sensitive!

Yes Windows 7's shell is still the same. Not only does it prevent you from creating files/folders too long, but if it happens to find one which exists you can't do anything with it (can't even delete it!) — John, May 14 '10 at 08:27
@John Thanks for the checking, good to know what 7 is capable of, although disappointing that it suffers the same restriction. — Stephen Nutt, May 14 '10 at 12:25

score 5 · Answer 4 · answered May 13 '10 at 09:54

Your own APIs should not hard-code a fixed limit on the path length (or any other hard limits); however, you shouldn't violate the preconditions of the system APIs in order to accomplish some task. IMHO, the fact that Windows limits the length of path names is absurd and should be considered a bug. That said, no I would suggest you not attempt to use the various system APIs other than as documented, even if that results in certain undesireable behavior such as limits to the maximum path length. So, in short, your view is completely fair; if the OS doesn't support it, then the OS doesn't support it. That said, you may want to make it clear to users that this is a limitation of Windows and not of your own code.

score 2 · Answer 5 · answered Jan 12 '11 at 12:39

One easy way how these files with long paths could be created even by software that does not support paths longer than MAX_PATH: through a file share.

Example:

"C:\My veeeeeeeeeeeeeeeeeeeeery looooooooooooooooooong folder" could be shared as "data". Users could then access that folder through the UNC path \\computer\data or (even shorter) through a drive letter (M:\) assuming that M: is mapped to \\computer\data.

This often happens on file servers.

score 1 · Answer 6 · answered Sep 25 '12 at 04:26

Paths often can be bigger than 260, one example would be when symlinks get nested and repeat over and over sometimes even on purpose. I think programmers should think about whether they want their program to handle these insanely large paths or not. IMO, 260 is PLENTY of space but thats just me. My answer to this is:

if you have to ask yourself so deeply about breaking the 260 char limit, then thats probably what you should do. We often look for confirmation when we are about to do something that we are unsure about...

I think the maximum path anywhere in the API is about 32k long but thats up to you. Back in the day that was a pretty big chunk of change (half of an entire memory segment!! sheesh!) but nowdays, in the segment-transparent addressing environment in which we live, where all memory is heaped together on the flat, 32k is nothin'... AFAIK paths wouldn't need to be that long unless you are using some fancy unicode language that requires lots of other characters, etc, etc.. we could blab about this all day but you get the idea. I hope this helps..... or hurts?

score 1 · Answer 7 · answered Jul 07 '13 at 17:48

I am doung some C programming and I was searching for a way to get the maximum length of a given filename, after a search for MAX_PATH I stumbled to this thread and after som thoughts on this matter and after reading the comments on this thread I have come to the following conclusion.

So I understand that NTFS support filenames up to 32.767 characters in length, however, according to knowledge FAT16 only support 11 character filenames, 8 + 3, so in reallity operating systems should have a function which our program can call to dertemine the maximum filename size, simply because all filesystems have different limitations including the length of the filename.

So the end conclusion must be that since us, the developers, don't know anything about which filesystem the data is going to be stored in, so therefore the only solution must be an try and error method.

score 0 · Answer 8 · answered Feb 16 '13 at 06:15

Not strictly an answer to your specific question, but it might help those who do need to handle long file names.

The Delimon library is a .NET Framework 4 based library on Microsoft TechNet for overcoming the long filenames problem:

Delimon.Win32.IO Library (V4.0).

It has its own versions of key methods from System.IO. For example, you would replace:

System.IO.Directory.GetFiles

with

Delimon.Win32.IO.Directory.GetFiles

which will let you handle long files and folders.

From the website:

Delimon.Win32.IO replaces basic file functions of System.IO and supports File & Folder names up to up to 32,767 Characters.

This Library is written on .NET Framework 4.0 and can be used either on x86 & x64 systems. The File & Folder limitations of the standard System.IO namespace can work with files that have 260 characters in a filename and 240 characters in a folder name (MAX_PATH is usually configured as 260 characters). Typically you run into the System.IO.PathTooLongException Error with the Standard .NET Library.

Should I deal with files longer than MAX_PATH?

8 Answers8

Linked