1

Is there a way to in a bash terminal mass recursive copy (cp -rf equivalent) from one subdirectory to another, preserving the directory structure but ONLY copying files/directories are version controlled by git?

The naive strategy would be...

cp -rf <targetDir> variantDir/; git add variantDir;

... but that would pick up a bunch of files that are data files dropped by the IDE, etc. I'm working with -- files which aren't currently version controlled in <targetDir>.

As background, I have the following directory structure:

${baseDir}/
${baseDir}/variantA

Basically, ${baseDir}/ contains all the original version controlled content, and ${baseDir}/variantA contains the content that differs for a targeted variant. For the original target folder ${baseDir}/variantA is ignored. For the second target the folder variantA is included, effectively superceding any content that matches other than the variantA part of the path.

i.e. if I had:

${baseDir}/someFolder/someStuff.cpp
${baseDir}/variantA/someFolder/someStuff.cpp

It would use the second path for the second target, and the first path for the first target.

Now I'm adding variantBand it's a full variant (unlike variantA that has some fall through to the base target's files). It's closer to variantA so I want to copy over everything in the base folder first, THEN copy over everything in the variantA subfolder to the variantB subfolder ... that way I have a new copy of all content (including the fall throughs) to version commit, with a preference for variantA file versions, when such overlap exists.

I would use the naive snippet above, but I have a more fundamental problem that my IDE has dropped *.xml files and other junk that git is somehow smart enough to flag as not uncommitted content (probably language specific), but which cp would ignore. I believe git would copy these files if EXPLICITLY asked to do, so, hence I want to exclude all the non-git controlled content, while using the general strategy stated prior.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
Jason R. Mick
  • 5,177
  • 4
  • 40
  • 69
  • 1
    Aren't you kind of re-inventing branching? – Benjamin W. May 23 '17 at 19:24
  • Dir structure is more or less out of my control to some extent for the code base I'm currently working on, but the reasoning behind it sort of makes sense. Traditional branching is used for some cases, while variant directories with a specialized build system is used to cut down on the redundancy while targeting simultaneously multiple targets... I'm kind of locked in, though, but yea, I get it's a weird case in some regards, but that's why I listed the simple version first as I feel the general question of how to `cp -rf` only version controlled content is pretty ubiquitous. – Jason R. Mick May 23 '17 at 19:30
  • I wouldn't expect any tool *other* than `git` (or one specifically designed to work with a Git repository) to be able to do this. – chepner May 23 '17 at 19:30
  • 3
    Why not use the `git archive` command to do this? It can pipe a tarball to its stdout, so pipe that to `tar -x -C /path/to/destination` and you're done. – Charles Duffy May 23 '17 at 19:34
  • I'd imagine the general strategy suggested for `git` controlled repos would also apply to `svn` using equivalent commands. My current best idea is `git log com/ford/theme/Resources/IconTheme=ford-gen2|wc -l` as that gives `0` for version controlled stuff, but some non-zero number of lines for version controlled stuff... so my working plan was to copy the dir structure, checking w/ that, then copy files only via recursive listing, again checking with that... but that's kind of a clunky way to do it... I was hoping for something more pleasing & smooth. – Jason R. Mick May 23 '17 at 19:35
  • 2
    https://stackoverflow.com/questions/15606955/how-can-i-make-git-show-a-list-of-the-files-that-are-being-tracked has commands to show all tracked files. – Benjamin W. May 23 '17 at 19:36
  • Charles, sounds like a potential solution... I'm unfamiliar with that particular command... so it only `tar`s the version controlled content? – Jason R. Mick May 23 '17 at 19:36
  • Correct, it only archives version-controlled content. – Charles Duffy May 23 '17 at 19:37
  • as @Charles said, ```git archive``` is your friend – eftshift0 May 23 '17 at 19:37
  • Ah, Benjamin that might be even better... agreed I could probably just use that directly... again not a `git` master by any means, so both those usages are new to me. – Jason R. Mick May 23 '17 at 19:37
  • Combine `git list-tree ...` with https://stackoverflow.com/questions/947954/how-to-have-the-cp-command-create-any-necessary-folders-for-copying-a-file-to-a and you have a simpler strategy than `git archive` as it doesn't rely on intermediates. – Jason R. Mick May 23 '17 at 19:42

4 Answers4

3

Why not just create a fresh work tree (which obviously will only have source-controlled files in it since it's generated from source control)?

There are a few ways to do this (with clones or work trees) but I would think

git worktree add -b variantB path/where/variantB/will/be/created master

would get you started with the relevant files (those under source control). Then you just copy path/where/variant/B/will/be/created/variantA over path/where/variant/B/will/be/created

And because of the -b option, you're already on a fresh branch - which is how you'll likely want to maintain this variation anyway since it's a full replacement. (For varientA it's debatable since you want variantA to keep up-to-date with baseline changes to the files it doesn't vary...)

But if you don't want to maintain a new branch for this, no big deal. Use either a clone or a worktree add command (without -b) to create a worktree image of variantB as above, but then move the resulting files into your default worktree under a .../variantB directory

Mark Adelsberger
  • 42,148
  • 4
  • 35
  • 52
2
git ls-files | tar Tc - | tar Cx /path/to/destination

this takes what's at your current spot in the worktree. To use a subdirectory of a commit use the git archive solution @Charles Duffy mentioned,

git archive master:path/to/subdir | tar Cx /path/to/destination

and to work with your currently-added-but-not-committed content you can subsitute $(git write-tree) for master above.

jthill
  • 55,082
  • 5
  • 77
  • 137
1

To copy only files under source control from variantA to variantB:

cd variantA
rsync --files-from <(git ls-files .) . ../variantB

This works by including only files returned by git ls-files.

Edit: the <() syntax is called process substitution.

Sjoerd
  • 74,049
  • 16
  • 131
  • 175
  • This seems like a good answer, but I'm on `Mintty` `BASH` which doesn't have `rsync` built in... is there an equivalent w/out it that you'd propose? `BASH` version is `GNU bash, version 4.3.46(2)-release`; `Mintty` version is `2.0.3` – Jason R. Mick May 23 '17 at 19:45
  • If this is on Windows, you may not have working process substitutions either. I don't remember if it extends to msys builds, but that functionality was broken on cycgwin builds of bash for a long, long time. – Charles Duffy May 23 '17 at 20:05
1

If you have GNU cp, you can use the --parents option:

mkdir variantB
cd variantA
cp --parents $(git ls-files .) ../variantB
Sjoerd
  • 74,049
  • 16
  • 131
  • 175
  • 1
    This will fail badly if any filenames contain spaces or could be interpreted as globs. Much safer to use `git ls-files -z .` and parse the NUL-delimited stream this emits. – Charles Duffy May 23 '17 at 20:02