Crawling GitHub with JGit

Question

I'm trying to crawl a GitHub Wiki with JGit.
When I try it with one URL, it worked perfectly fine. Then I tried it with another random URL and got an error.
Please see the extract of my code:

import java.io.File;
import java.io.IOException;

import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;

public class Main {
    // with this URL I get an error
    String url = "https://github.com/radiant/radiant.wiki.git";
    // this URL works
    // String url = "https://github.com/WardCunningham/Smallest-Federated-Wiki.wiki.git";

    public static void main(String[] args) {
        Main m = new Main();
        m.jgitTest();
        System.out.println("Done!");
    }

    public void jgitTest() {
        try {
            File localPath = File.createTempFile("TestGitRepository", "");
            localPath.delete();
            Git.cloneRepository().setURI(url).setDirectory(localPath).call();
        } catch (IOException | GitAPIException e) {
            System.err.println("excepton: " + e.getMessage());
            e.printStackTrace();
        }
    }
}

This is the stack trace:

Exception in thread "main" org.eclipse.jgit.dircache.InvalidPathException: Invalid path (contains separator ':'): How-To:-Create-an-Extension.textile
    at org.eclipse.jgit.dircache.DirCacheCheckout.checkValidPathSegment(DirCacheCheckout.java:1243)
    at org.eclipse.jgit.dircache.DirCacheCheckout.checkValidPathSegment(DirCacheCheckout.java:1225)
    at org.eclipse.jgit.dircache.DirCacheCheckout.checkValidPath(DirCacheCheckout.java:1185)
    at org.eclipse.jgit.dircache.DirCacheCheckout.processEntry(DirCacheCheckout.java:311)
    at org.eclipse.jgit.dircache.DirCacheCheckout.prescanOneTree(DirCacheCheckout.java:290)
    at org.eclipse.jgit.dircache.DirCacheCheckout.doCheckout(DirCacheCheckout.java:408)
    at org.eclipse.jgit.dircache.DirCacheCheckout.checkout(DirCacheCheckout.java:393)
    at org.eclipse.jgit.api.CloneCommand.checkout(CloneCommand.java:236)
    at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:127)
    at Main.jgitTest(Main.java:21)
    at Main.main(Main.java:13)

If you visit the wiki page of the URL that doesn't work (https://github.com/radiant/radiant/wiki), you will find this page: How To: Create an Extension.
The title of this page is the cause of the error: Invalid path (contains separator ':'): How-To:-Create-an-Extension.textile.

I assume I need to escape all output.

score 6 · Accepted Answer · answered Sep 10 '14 at 18:54

6

I suppose you are on windows. You can't create a file on windows having the ":" in the name. JGit should handle it somehow, so I suppose this is a bug in JGit.

answered Sep 10 '14 at 18:54

Simone Gianni

11,426
40
49

Yes, I'm using Windows. Is there any chance to escape the file names? – Evgenij Reznik Sep 10 '14 at 18:59
Git itself also rejects such pathes on Windows, see e.g. http://superuser.com/questions/655318/git-error-invalid-path – centic Sep 12 '14 at 05:44

score 3 · Answer 2 · edited May 23 '17 at 12:31

3

I had the same problem with pure git, and this answer helped me:

git config core.protectNTFS false

edited May 23 '17 at 12:31

Community

1
1

answered Feb 10 '16 at 17:33

Berci

544
1
7
10

Crawling GitHub with JGit

2 Answers2