SVN: Sparse update to create only trunks

Question

I have looked around at several solutions to checking out only svn trunks. Following the redbook we have a traditional svn set up, i.e.

/
|-- /branches
|-- /tags
`-- /trunk
    |-- foo
    `-- bar

The thing is we have 100's of projects, all maven modules. Each project follows the structure above.

I would like to go to the root of our repo and check every project out such that I have a structure similar to:

A/
 `-- /trunk
    |-- foo
    `-- bar
B/
 `-- /trunk
    |-- foo
    `-- bar
C/
 `-- /trunk
    |-- foo
    `-- bar
... etc...

I have achieved this using TortoiseSVN and using the option to Update To Revision -> Choose Items and deselecting all the tags and branches.

This gives the following style of output:

Sparse update tags, depth 'Exclude'
C:\svnrepo\A\tags
Sparse update branches, depth 'Exclude'
C:\svnrepo\A\branches
Sparse update trunk, depth 'Fully recursive'
C:\svnrepo\A\trunk

What I would like to do is not have to click through all these tags and branches checkboxes. Does TortoiseSVN have a way to achieve this? Does anybody have a script that can run on windows and run the appropriate command line(s) to exclude every tag and branch from every project? I have looked here but it does not make much sense:

http://svnbook.red-bean.com/en/1.5/svn.advanced.sparsedirs.html

Instead of using TortoiseSVN, would you mind using the command line `svn` command? The command line isn't that complex and is much easier for creating these type of scripts. — David W., May 11 '15 at 16:01
David, thanks for the reply. I've looked at the redbook for sparse dirs as listed in my post and its just not clear to me what the commandline should look like. Its clear there is a way to do this as Tortoise is doing it, but how to iterate through the various directories and "Exclude" or be "Fully recursive" is beyond my current understanding. Any guidance would be a help. — theINtoy, May 11 '15 at 16:24
You need a mixture of either PowerShell or Batch to go through the data. You can use `svn ls %REPO%` to list your projects. I would checkout `svn co --depth=immediates %REPO%` to do the initial checkout of all projects, then use a `for` loop to go through all of the directories and do a `svn up --set-depth=infinity %PROJ%/trunk` to get the complete `trunk` of each project. I'll work out the batch script, and post an answer. — David W., May 11 '15 at 18:26

score 1 · Answer 1 · edited May 12 '15 at 10:27

Okay, not having a PC, but a Windows bath script would be something like this:

svn co --depth=immediates %REPO% projects
cd projects
for /D %%d in (*) do svn up --set-depth=infinity %%d/trunk

I don't have your exact setup, so I can't say whether this will work, or you'll have to cd into %%d first, then do a svn --set-depth=infinity trunk and then cd .. to get to the next project. However, this will give you an idea how you combine Windows batch scripting with the svn command line client to do things you can't easily do through a GUI client.

Of course, the question is about why you want to check out out each and every project (100s of them) unless you're actually working on all the hundreds of projects. This could take a really, really long time to process. By the time you actually get to a particular project to do work, it'll be out of date anyway.

You might be better using sparse checkouts to checkout each project (and maybe the trunk) and then when you actually need to work on a project, to do a `svn up --set-depth=infinity on the project's trunk directory. Maybe something like this might work better:

svn co --depth=immediates %REPO% projects
cd projects
for /D %%d in (*) do svn up --set-depth=empty %%d/trunk

Then, you can do an svn update --set-depth=infinity when you actually work on the project.

David, thanks again. Your answer works for the question I've asked, thanks! I did make one small change: svn co --depth=immediates %REPO% projects. As you point out I didnt quite list the structure and its not as flat as every project in root, there are many projects grouped in a parent folder off root and then a further few projects have a another grouping folder, so we have a maximum depth of a/b/c/pom.xml for some projects. Is there a way to test if the %%d contains a pom.xml or something? — theINtoy, May 12 '15 at 10:22
From my comment, "we have a maximum depth of a/b/c/pom.xml" should read "we have a maximum depth of a/b/c/trunk/pom.xml" — theINtoy, May 12 '15 at 11:16
In Unix, you could execute the command `svn ls $d/pom.xml > /dev/null 2>&1` where `$d` is the name of the project. You could then check the exit status of that command. If it's zero, the `$d/pom.xml` exists. If not, you know that `$d/pom.xml` doesn't exist. I'm not 100% sure how to do this with Windows. Maybe run the command as `svn cat %%d/pom.xml > NUL 2> NULL` and then check it via `if errorlevel 1`. — David W., May 12 '15 at 19:39

theINtoy · Accepted Answer · 2015-11-27T14:40:29.783

In answer to my own question. As a Java dev, my shell skills are not top notch so decided eventually to drop in to Java.

Using SVNKIT which is not that quick, I was able to create this one method. I have kept it all in one method so as to post it here and hope it will help some others at least as a pointer if not a solution. It is built against Java 7.

...

public void doRootCheckout(Path workingCopyDirectory) throws SVNException, IOException {
    // Following advice from: http://stackoverflow.com/a/14386871/1279002
    final SvnOperationFactory svnOperationFactory = new SvnOperationFactory();

    // USERNAME/PASSWORD are the svn username and passwords. 
    // Clearly you will need to use your own!                 
    svnOperationFactory.setAuthenticationManager(SVNWCUtil.createDefaultAuthenticationManager(USERNAME, PASSWORD.toCharArray());

    try {
        final SvnCheckout checkout = svnOperationFactory.createCheckout();
        checkout.setSingleTarget(SvnTarget.fromFile(workingCopyDirectory.toFile()));

        // repoUrl is the url of your SVN repository and in this 
        // example is a UTF-8 string, e.g. http://my.repo.com/java
        SVNURL url = SVNURL.parseURIEncoded(repoUrl);
        checkout.setSource(SvnTarget.fromURL(url));
        checkout.setAllowUnversionedObstructions(false);

        // Our local clients are fixed at the 1.7 svn format so had to set this here
        checkout.setTargetWorkingCopyFormat(ISVNWCDb.WC_FORMAT_17);

        // This solution is somewhat inefficient as it first
        // checks out all the code from the root of the repoUrl
        // including all tags and branches.
        // This can take a long time...
        checkout.setDepth(SVNDepth.INFINITY);
        checkout.run();

        // When the full checkout is completed we
        // need to iterate through all the directories 
        // and exclude all tags and branches... 
        // This also can take a long time...        
        Files.walkFileTree(workingCopyDirectory, new SimpleFileVisitor<Path>() {
            @Override
            public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) {

                if (dir.endsWith("tags") || dir.endsWith("branches")) {
                    System.out.println("Excluding: " + dir.toAbsolutePath().toString());
                    SvnUpdate svnUpdate = svnOperationFactory.createUpdate();
                    svnUpdate.setDepth(SVNDepth.EXCLUDE);
                    svnUpdate.setDepthIsSticky(true);
                    svnUpdate.setSingleTarget(SvnTarget.fromFile(dir.toFile()));
                    try {
                        svnUpdate.run();
                    } catch (SVNException e) {
                        e.printStackTrace();
                    }
                }
                return FileVisitResult.CONTINUE;
            }
        });

    } finally {
        svnOperationFactory.dispose();
    }

}

...

SVN: Sparse update to create only trunks

2 Answers2