166

When I attempt to checkout a repository from github I get the error:

error: invalid path 'configs/perl-modules/DIST.64/perl-HTML-Tree-1:5.03-1.el6.noarch.rpm'

I suspect the issue is that the path contains a : which is illegal on Windows.

After researching the error, I've found 2 possible answers:
1) Change the path on the repository file. Unfortunately, this is is a team resource and can not be fixed in the foreseeable future.
2) Use sparse-checkout. I've tried this with no effect as evidenced in the following:

$ git clone -n git@github.com:XXXXXX/deploy.git
Cloning into 'deploy'...
remote: Enumerating objects: 57, done.
remote: Counting objects: 100% (57/57), done.
remote: Compressing objects: 100% (49/49), done.
remote: Total 86457 (delta 10), reused 22 (delta 8), pack-reused 86400
Receiving objects: 100% (86457/86457), 1.50 GiB | 4.73 MiB/s, done.
Resolving deltas: 100% (59779/59779), done.
$ cd deploy/
$ git config core.sparsecheckout true
$ echo www >> .git/info/sparse-checkout
$ git checkout centos6
error: invalid path 'configs/perl-modules/DIST.64/perl-HTML-Tree-1:5.03-1.el6.noarch.rpm'
error: invalid path 'configs/perlbrew/perls/perl-5.24.1/man/man3/App::Cpan.3'
.
. (repeat for many files)
.

This was done with Git for Windows "git version 2.28.0.windows.1". I have also tried both types of line endings and using various version of .git/info/sparse-checkout such as:

/*
!/configs/perl-modules
!/configs/perlbrew/perls/perl-5.24.1/man/man3

Checkout works fine on Linux, MacOS and WSL, only problem is that my IDEs don't work there. Why isn't sparse-checkout working on Windows. Is there any other possibilities?

wdtj
  • 4,554
  • 3
  • 17
  • 20
  • Other things I've tried are changing line endings, leading /, \ instead of / (after all, this is windows), and adding a * at the end. I also tried the new svn sparse-checkout init and svn sparse-checkout add & set commands. Still no joy. – wdtj Sep 04 '20 at 01:40
  • 1
    Create a fork, clone it on a real computer, rename the problematic files, commit, push? Then you should be able to do a shallow checkout on your hapless Windows system. – tripleee Sep 04 '20 at 13:40
  • 1
    It might also be a blank at the end of a folder name. Happened to me... And hard to detect! – Prof.Chaos Jul 15 '21 at 05:28
  • If you can do it in WSL shouldn't you be able to view things there, delete/rename anything problematic, and then switch to W10 "normal"? – mike rodent Jan 28 '22 at 19:30
  • If you can do it in WSL shouldn't you be able to view things there, delete/rename anything problematic, and then switch to W10 "normal"? I just tried this and it seems to have worked. Having switched branches in WSL I then went back to Eclipse IDE and things (seem! to be) working OK. However, when I want to switch *back* to that problem branch from a "healthy" branch I have to do it in WSL! NB the things with problem paths in my case are in commits buried deep in the repo. As a low-level gitter I have no idea how I would go about finding and deleting them (they are superfluous in my case). – mike rodent Jan 28 '22 at 19:49
  • As @Prof.Chaos said, it can be a space at the end of a directory OR file name. [Here is an example](https://imgur.com/a/mNL3rr0). – Dario Fumagalli Nov 16 '22 at 03:37
  • This is due to Windows NTFS Alternate data streams https://owasp.org/www-community/attacks/Windows_alternate_data_stream TL;DR: echo "foo" > file.txt:stream now file.txt has an alternate data stream called "stream" with contents "foo". –  Feb 06 '23 at 02:56
  • I had this same error, but with a file that had several `|` in the file name (youtube shortcut created by a MacOS user). I fixed it by renaming the file on GitHub to remove those characters. I didn't try to recreate my clone with `git clone -c core.protectNTFS=false ` as suggested in the comments of one of the answers below. – Fuhrmanator May 06 '23 at 13:47

18 Answers18

235

After I opened an issue on the git-for-windows bug tracker (https://github.com/git-for-windows/git/issues/2803), I found that my issue had already been filed as https://github.com/git-for-windows/git/issues/2777. That issue suggested that I need to set another git flag:

git config core.protectNTFS false

This (#2777) indeed contains a bypass for the my problem. I hope the git or git-for-windows (who were very responsive) come up with a better warning message, or even a true fix like a filepath mapping scheme.

Note that this is only an issue when using sparse-checkout with windows.

SwissCodeMen
  • 4,222
  • 8
  • 24
  • 34
wdtj
  • 4,554
  • 3
  • 17
  • 20
  • 3
    Interesting. I had not heard of `core.protectNTFS` doing this, but searching through the Git source code, there it is. The comment around it says it's looking for double colon but it rejects even a single colon, and all this rejecting happens well before any sparse checkout step. – torek Sep 04 '20 at 18:07
  • 1
    Issue 2803 just got updated: https://github.com/git-for-windows/git/issues/2803#issuecomment-689621560 – VonC Sep 10 '20 at 07:20
  • @wdtj were u able to get this working ? I am having the same issue and opened this https://stackoverflow.com/questions/67926498/how-to-use-git-sparse-checkout-in-2-25-and-not-checkout-everything?noredirect=1#comment120202716_67926498 a couple of days ago. Came across ur post today, tried `core.protectNTFS`, but no luck – mittal Jun 18 '21 at 10:34
  • No. Ended up making a Linux VM to do my development on. – wdtj Jun 18 '21 at 23:00
  • Yes it worked for the same error when i got while cloning repo , after executing command "git config core.protectNTFS false " execute "git reset" and "git checkout * " – Surendra Babu Parchuru Aug 02 '21 at 11:33
  • 1
    This answer does not work as of September 8, 2021. –  Sep 08 '21 at 20:15
  • It works for me. I had issues when doing `git checkout` (and with `git clone`). The fix worked for me – mihca Dec 25 '21 at 18:21
  • Note that `core.prtectNTFS` settting was created in order to close a security vulnerability. So be thoughtful about the situations where you disable the setting. Also see https://confluence.atlassian.com/bitbucketserverkb/error-invalid-path-during-git-clone-to-windows-client-1085186345.html – matt wilkie Dec 26 '21 at 17:22
  • 1
    This worked for files with `aux` in the name, but didn't help for files with a backslash in the name (there is such a file in the AOSP source). – ymett Jul 31 '22 at 10:13
  • 2
    The windows NTFS naming rules are here https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file#naming-conventions I had a directory name 'aux'. – phreed Sep 30 '22 at 16:00
  • Worked for me...it complained about a firebase.json file...I don't understand. But thanks! – OtaconKiko Nov 08 '22 at 10:24
  • Setting this appears to allow the checkout to succeed, but all the problematic files are renamed on checkout resulting in an unclean checkout in all cases. Any way I can stop that? – Chris Withers Jan 18 '23 at 15:20
  • @ChrisWithers you would just have to commit with ntfs-compatible name. collaboration is broken if a file system cannot store the filename and the environment is windows & linux users ... in my case i had a wildcard char in a filename for a wildcard certificate.. can say microsoft is behind but i can also say i am just too binding literal? [/changes] – Kalen White Feb 15 '23 at 16:36
  • @KalenWhite - no can do, I'm afraid: the names need to stay as they are on linux. – Chris Withers Feb 15 '23 at 22:38
  • In our case, it was a filename "con.js" that it was objecting to. Linux and macOS developers had no such issue. This workaround worked. Also did the extra steps @sn0maid3n mentioned in the other toppish answer here. – storsoc Apr 04 '23 at 15:27
  • This answer worked for me as of June 2023 . Thanks for the solution . – chandiprasad Jun 12 '23 at 07:03
65

I have experienced a similar problem, trying to checkout a repository from GitHub that contained files with a ":" in the name, on a Windows. (example file name that caused the problem: "test-img.jpg:Zone.Identifier"). The repo downloaded, but the files were not showing up in the folder.

I found that running git config core.protectNTFS false solved my issue, but only if I took some steps before and after running it. The entire process went like so:

  1. Clone the Git repository locally;
  2. 'cd' into the local Git repository folder that has just been cloned;
  3. Run git reset
  4. Run git config core.protectNTFS false
  5. Run git checkout (just git checkout, no * at the end of command).

After that, I could see the files. Granted, some extra things downloaded from Git that are usually omitted, but it wasn't a big deal in my case.

Dharman
  • 30,962
  • 25
  • 85
  • 135
sn0wmaid3n
  • 659
  • 4
  • 5
  • Thank you very much for your solution, which almost got me there. I want to add on the "After that, I could see the files" though: In my case, I had a file named "foo:bar.txt", which after your steps, became "foo". I noticed, however, that the "foo" file was empty (0 bytes). So, additionally, I (1) downloaded my original repo as a zip file, (2) renamed "foo:bar.txt" in the zip file to "foo_bar.txt", (3) replaced "foo" with that, and (4) commited. – simon Oct 08 '21 at 13:44
  • 2
    wanted to add, you can set a config option on a new repo while cloning with `-c` ([docs](https://git-scm.com/docs/git-clone#Documentation/git-clone.txt--cltkeygtltvaluegt)). i used `git clone -c core.protectNTFS=false ` and it worked for me – starwarswii Jul 20 '22 at 14:22
  • 1
    Can this config be set globally so this does not need repeating? `git config --global core.protectNTFS false` – jimh Jul 27 '22 at 20:58
  • 1
    @jimh, this is not a good idea, as it will make your system more vulnerable to hacks. See Edward Thomson's comment under edocollado's answer. – sn0wmaid3n Jul 30 '22 at 23:43
16

In our case there was a filename aux.go and windows doesn't allow the creation of a file that has the word aux

enter image description here

Read more about this issue here

Rengas
  • 593
  • 1
  • 6
  • 25
12

For those who want just the commands to workaround the issue:

git clone --sparse -c core.protectNTFS=false -n <repo-URL>
git sparse-checkout add "\!<pattern1>" "\!<pattern2>"
git checkout <branch>

Patterns are relative to repo root and can use *

Roman Orekhov
  • 330
  • 2
  • 6
8

Addendum - git version 2.34.1.windows.1 & others(?)

Regrettably, git config core.protectNTFS false turned out not to be enough; the contents of the colon-carrying filenames are lost (filesize = 0).

Solution

TL;DR

git diff ec28c8ddd5f8c83d11604bcae69afb46d79b1029 > p.patch
patch -R -f -i p.patch
git add *
git commit 

Elaboration

Turns out git config core.protectNTFS false does work, insofar as to not produce fatal errors on git checkout anymore.

However, git now will produce filenames clipped to the colon and with zero content.

E.g. Writing-Bops:-The-Bebop-Schema-Language.md (~9KB) --> Writing-Bops (0 KB)

To fix this, we need to get a copy of the original offending file(s)' content another way, so we can restore it.

Conditions / Assumptions

  • This assumes you cannot or will not use a sparse clone for some reason.
  • Ditto on git apply-filter and other techniques to 'permanently rewrite' git history, f.e. when you are tracking a third-party git repo.
  • You're running Windows, using NTFS storage, git-for-windows with bash as your shell and have a patch.exe available (patch --version should report something like "GNU patch 2.7.6")

(In our case, we were messing about with a github wiki clone and ran into this issue of filenames containing colons. Of course, we wanted to fix this in place, without going the extra 'sparse clone' or WSL mile.)

Turns out we can get the missing content after we did

git config core.protectNTFS false
git checkout <hash>

by running patch. (BTW: TortoiseGit would hang forever if you tried to diff/compare these commits!)

Use this next command to get a patch file with all the missing changes. If you have multiple files with colons or other trouble, all the missing content will be listed in the patchfile: one patchfile to catch it all!

git diff ec28c8ddd5f8c83d11604bcae69afb46d79b1029 > p.patch
# ^^^^ reference the git hash with the offending original file(s)

Now that you have a patchfile, you can apply it to the current working directory: it must be applied in reverse (-R):

patch -R -f -i p.patch

If you forget -R, patch will ask (answer [y]es); if you do specify -R patch will yak even more, so -f (force) is in order to shut up patch and just do the job.

This should list one or more files being patched as a result, e.g.

$ patch -R -f -i p.patch
patching file Writing-Bops:-The-Bebop-Schema-Language.md

Note the colon in that filename: turns out GNU patch on windows (at least v2.7.6) uses a Unicode homoglyph to simulate a colon in a filename. See also further below.

You now have the original content of the colon-ed files in your working directory and you're now ready to add these files to the git index and commit them as usual:

Warning: you'll probably need to cleanup (delete) the empty clipped filenames produced by your earlier git checkout before proceeding!

Note: if you don't like the homoglyphed filename patch -i assigned to the missing content, you can change it to anything you like before committing the result.

git add *
git commit 

Verifying results

When you did everything right, that last commit should list the colon-ed file as renamed as you did not change the content and git commit should thus have detected the "file rename action" as-is.

Extra: replacing colon with a homoglyph

I found several Unicode homoglyphs that look more or less like a colon, but are considered legal in NTFS filenames.

After a bit of experimentation, I decided on using as I wanted to keep the github wiki page I was fiddling with intact as much as possible.

Generally, I would discard the colon entirely, or replace it with one or more hyphens, but in the case of wiki MarkDown pages, that decision can go the other way.

Ger Hobbelt
  • 1,126
  • 1
  • 11
  • 16
7

I suspect the issue is that the path contains [colon character] : which is illegal on Windows.

That is in fact the problem.

[sparse checkout with] !configs/perlbrew/perls/perl-5.24.1/man/man3

The pathname being complained-about here is:

configs/perl-modules/DIST.64/perl-HTML-Tree-1:5.03-1.el6.noarch.rpm

which does not begin with configs/perlbrew/, much less the full to-be-skipped path.

You can probably work around this by (painfully) enumerating all the invalid file names. Git needs a better general mechanism for this, though.

torek
  • 448,244
  • 59
  • 642
  • 775
  • While you are correct that the paths didn't match, the errors (which I truncated for brevity) are occurring on both paths /configs/per-modules/ and /configs/perlbrew/. I updated the sparse-checkout file with both paths and still get the errors. Good try though. I'll update the description with a bit more. – wdtj Sep 04 '20 at 01:30
  • Might be a typo but you have `per-modules` in the comment above and in the edit, did you mean `perl-modules`? – torek Sep 04 '20 at 01:47
  • Your right, but that only effects one of many file errors. Thanks. – wdtj Sep 04 '20 at 13:18
4

I discovered one option that had not been mentioned. Use WSL to checkout the files.

I ran git clone in WSL/Bash instead of PowerShell/CMD. WSL handles fixing the filenames transparently, even though they are still stored on NTFS and I open them using VS Code for Windows. I was able to modify the files in Windows, and push my changes on Windows even.

pseudosavant
  • 7,056
  • 2
  • 36
  • 41
3

In GitHub: Going to the Repo URL (https://github.com/UserNameHere/RepoNameHere), then Clicking "Code" > "Download ZIP" and then extracting the .zip file in Windows was successful for me (I had to skip invalid files)

enter image description here

enter image description here

Jesus is Lord
  • 14,971
  • 11
  • 66
  • 97
2

before using git clone

execute this command:

git config --global core.protectNTFS false
edocollado
  • 471
  • 6
  • 7
  • 8
    Absolutely do not do this. This will not allow you to write files with special characters like a `:`. This will only allow people to write into your `.git` directory by using NTFS-specific hacks. This opens you up to attacks and does not solve the problem. – Edward Thomson Nov 02 '21 at 19:55
1

If none of the other solutions work, the problem might be the version of git. I used version 2.31.1 and the error was shown, but then I tried it with the 2.15 and it worked fine. But for me the error was with the git clone command.

flyingdutchman
  • 1,197
  • 11
  • 17
joca1128
  • 23
  • 5
1

If operating with different OS eg linux vs windows or Mac Os vs Windows. Check characters in the file path. In my case I saved items on my mac and committed, when pulling on my windows machines, file path error accured.

Windows states that

A filename cannot contain any of the following characters: \ / : * ? " < > |

But mine contained "*" in the image names saved. So just commit new names which are valid for the different OS your team is using. Or frustrate yourself more by looking for work-arounds

Simple right.

Marshall Fungai
  • 176
  • 1
  • 6
1

In my case the file name had an extra space at the end. On Windows this was causing an issue.

I was able to fix this issue by removing this space. As I was not able to clone the project on Windows, this change has been done directly on where our repository was (which is GitHub in our case).

Kris Stern
  • 1,192
  • 1
  • 15
  • 23
0

I have been like you , your filename is incorrect or not reserved , just rename your file

  • 3
    That would be a most reasonable solution, but read the question carefully: "Unfortunately, this is is a team resource and can not be fixed in the foreseeable future". They are unwilling to change the file name in the short term. – Daemon Painter Dec 21 '20 at 11:07
0

For those having a similar error:
Personally, the problem was using " in a file name on Linux and trying to pull on a Windows 10 machine.

gberth
  • 467
  • 5
  • 13
0

In my case, the repo contains files with invalid characters in Windows, like [ or ].

Shaokun
  • 78
  • 1
  • 7
0

Once I supplied a target path to the git command it stopped looking for c:/program files/git/src This failed: git log --pretty=email --patch-with-stat --reverse --full-index --binary -- /src/pathtofile

This works: git log --pretty=email --patch-with-stat --reverse --full-index --binary -- ./src/pathtofile

Giora
  • 1
  • 1
    As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Mar 01 '22 at 19:14
0

You might wanna check on file naming. Windows does not allow some characters at the end of the file name eg "?", ":". Delete / rename them well and try again. PS: spaces aren't allowed either.

  • 1
    This seems to already be considered in other answers, can you please provide an additional piece of information or add this as a comment? – aaossa Mar 06 '22 at 01:03
0

If this is a case of unavailable support for symbolic links, try to reinstall your Git for Windows with "Enable symbolic link" option enabled.

More details here: Git symbolic links in Windows