12

Is there a robust way to do a recursive depth-first git submodule foreach command? I am using the foreach --recursive command which does the job, except it is breadth-first. This is a problem because if I have the following structure:

  • A
    • B
  • C

And I have commits in all three, a foreach --recursive add -A && git commit ... will hit A, B, C, which is problematic if I want the supermodule to capture the commits of B at that time.

I found this discussion from 2008, but it does not look like any of the suggested features are in the current version of Git that I have (1.7.9.5).

I wrote a small bash function to do this (excuse the shorthand naming):

function git-sfed() { git submodule foreach "git submodule foreach '$*' && $*"; }

And testing it with the following fanciful command seems to work:

git-sfed 'python -c "import sys; print sys.argv" $path'

Does this command seem robust, or are there other common existing methods?

eacousineau
  • 3,457
  • 3
  • 34
  • 37
  • Note: some commands now are aware of submodule: for instance, `git grep -e "bar" --recurse-submodules` is available with Git 2.12: http://stackoverflow.com/a/41788645/6309 – VonC Jan 22 '17 at 08:00

3 Answers3

18

You can try this

git submodule  foreach --recursive  |  tail  -r | sed 's/Entering//' | xargs -I% cd % ; git add -A \& git commit

This list (recursively) all the submodules , then reverse the list, tail -r so you get the directories in the order you want (child first), enter the directory and do what ever you want in it.

mb14
  • 22,276
  • 7
  • 60
  • 102
  • Interesting, more complete answer than mine. +1 – VonC Feb 13 '13 at 15:19
  • 1
    Thanks for this one! Unfortunately, though, I did not have the `-r` option (Ubuntu), but through [this post](http://stackoverflow.com/a/742485/170413) there's the `tac` command. So with a different flavoring of xargs, came up with another variant: `git submodule foreach --recursive | tac | sed 's/Entering //' | xargs -n 1 bash -c 'cd $1 && git status' _` – eacousineau Feb 13 '13 at 17:50
  • I've been trying out my method some, but I believe I get an error if I want to run a commit on all submodules and then the supermodule. I will be testing your method later to see if it is more robust. I will come back to it once I test it out. – eacousineau Feb 16 '13 at 01:46
  • Sorry it took a while, but I've been having trouble with by yours and my command when trying to put in single quotes (kinda sucks not being able to write them) - escaping with multiple levels of bash commands is a little confusing haha. Right now, I'm figuring the next best option is to try and submit a patch for git-submodule, so I will be trying that out. – eacousineau Mar 04 '13 at 08:15
3

I didn't find any other way than your function to perform a depth-first foreach command.

The test would be to check if it does achieve recursive for a depth of more than one.

A
  B
    D
  C

I've been having trouble with by yours and my command when trying to put in single quotes (kinda sucks not being able to write them) - escaping with multiple levels of bash commands is a little confusing.

This (quotes issue) should be simplified in Git 1.9/2.0 (Q1 2014), with commit 1c4fb13 from Anders Kaseorg (andersk):

'eval "$@"' creates an extra layer of shell interpretation, which is probably not expected by a user who passes multiple arguments to git submodule foreach:

 $ git grep "'"
 [searches for single quotes]
 $ git submodule foreach git grep "'"
 Entering '[submodule]'
 /usr/lib/git-core/git-submodule: 1: eval: Syntax error: Unterminated quoted string
 Stopping at '[submodule]'; script returned non-zero status.

To fix this, if the user passes more than one argument, execute "$@" directly instead of passing it to eval.

Examples:

  • Typical usage when adding an extra level of quoting is to pass a single argument representing the entire command to be passed to the shell.
    This doesn't change that.
  • One can imagine someone feeding untrusted input as an argument:
    git submodule foreach git grep "$variable"

That currently results in a nonobvious shell code injection vulnerability.
Executing the command named by the arguments directly, as in this patch, fixes it.

Since Git 2.21 (Q2 2017), you have git grep -e "bar" --recurse-submodules

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • I tested out both mb14's and my techniques and they both seem to work. I posted an [example](http://pastebin.com/eHeYiya7) on pastebin. – eacousineau Feb 13 '13 at 21:07
0

Your method actually will only do two levels of depth. To implement the full recursion, you'll want to call your script itself recursively. Something like the below should do...

#!/bin/bash
SCRIPT=$(realpath "$0")
SCRIPTDIR=$(dirname "$SCRIPT")
git submodule foreach "$SCRIPT '$1' && $1"
lmonninger
  • 831
  • 3
  • 13