2606

How do I clone a git repository so that it also clones its submodules?

Running git clone $REPO_URL merely creates empty submodule directories.

Mateen Ulhaq
  • 24,552
  • 19
  • 101
  • 135
  • 3
    @LiamCrowley , the parent (hosting, containing) repo might depend on a particular version of the submodule for a variety of reasons. For example, the maintainers of the host repo might not be ready to deal with updates just yet. – arkadianriver May 21 '21 at 12:13
  • if your repo already exists then you do git submodule init then git submodule update --init potentially with the --remote flag too e.g. https://stackoverflow.com/a/74999430/1601580 – Charlie Parker Jan 03 '23 at 23:01
  • 1
    for ref, do `git clone --recurse-submodules --remote-submodules ` – Charlie Parker Jan 03 '23 at 23:19

24 Answers24

3660

With version 2.13 of Git and later, --recurse-submodules can be used instead of --recursive:

git clone --recurse-submodules -j8 git://github.com/foo/bar.git
cd bar

Editor’s note: -j8 is an optional performance optimization that became available in version 2.8, and fetches up to 8 submodules at a time in parallel — see man git-clone.

With version 1.9 of Git up until version 2.12 (-j flag only available in version 2.8+):

git clone --recursive -j8 git://github.com/foo/bar.git
cd bar

With version 1.6.5 of Git and later, you can use:

git clone --recursive git://github.com/foo/bar.git
cd bar

For already cloned repos, or older Git versions, use:

git clone git://github.com/foo/bar.git
cd bar
git submodule update --init --recursive
Mathias Bynens
  • 144,855
  • 52
  • 216
  • 248
  • 197
    Is there any way to specify this behavior as default in your git repository, so that less-informed cloners will automatically get an initialized submodule? – NHDaly Feb 20 '13 at 06:37
  • 21
    @NHDaly Sadly, no. (Not that I know of, at least.) – Mathias Bynens Feb 20 '13 at 07:25
  • 10
    And logically thinking git clone --recursive will also populate any submodules of a submodule, right? – jayarjo Apr 10 '13 at 17:31
  • 3
    @NHDaly it seems not: http://stackoverflow.com/questions/4251940/retrospectively-add-recursive-to-a-git-repo – Ciro Santilli OurBigBook.com May 04 '13 at 13:28
  • 3
    you may want to consider git subtree https://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/ It solves the problem in @NHDaly comment – piccolbo Jan 02 '14 at 22:03
  • 2
    @NHDaly sure, just add this into your `~/.gitconfig` under the `[alias]` section: `cloner = clone --recursive` – slf Aug 25 '14 at 17:42
  • 2
    @slf While that’s helpful, it doesn’t answer @NHDaly’s question. `.gitconfig` settings are global, not just for a single repository. – Mathias Bynens Aug 25 '14 at 20:10
  • @mathius-bynens ah you are correct sir. I misread the question. – slf Aug 26 '14 at 00:15
  • 1
    After doing either of these, it's wise to switch sources to the master branch (or any branch) since your repo will be sort of headless. From the top level repo -- ```git submodule foreach git checkout master``` – 4Z4T4R Sep 05 '14 at 15:14
  • 5
    @toszter: is it so wise? What if the holding repo needs a version of a submodule that is *not* `master`? – Gauthier Apr 29 '15 at 12:27
  • @Gauthier - Thank you... if it's _any branch_ other than master, as mentioned, substitute `master` with your preferred branch name _other_ than `master`. – 4Z4T4R May 01 '15 at 02:27
  • 3
    @toszter Even so, the correct commit is not necessarily at the tip of the branch. – Gauthier May 01 '15 at 08:39
  • you can use `git submodule update --depth 10 --recursive --init` to shallow the repo – yuxuan Feb 11 '16 at 03:31
  • This doesn't work with git 2.7, but worked with git 2.9 – kmario23 Aug 25 '16 at 03:14
  • Thanks! I had forgotten to actually retrieve the submodule dependency contents with `git submodule update --init --recursive` before running `sh -c 'cd cmd/ios7crypt && go install'` – mcandre Oct 23 '16 at 22:20
  • 1
    I did `git submodule update --init --recursive` but all the submodules are in dechated head state, I have to go into each submodule and checkout master. Does anybody know why? – VaTo Mar 23 '17 at 02:46
  • 2
    I have git 1.9.1 and `git clone --recursive -j8 git@bitbucket.org:repoToClone` does not work. It tells me `error: unknown switch `j' ` If I remove the `-j8` it works smoothly – desmond13 Apr 06 '17 at 14:17
  • 1
    As it seems, the flag name was updated to `--recurse-submodules` – Eugene Sep 10 '17 at 11:57
  • 18
    I think I come back to this answer once a month... Why don't git just ask upon cloning if it should also download the submodules? – rgoliveira Oct 12 '17 at 21:44
  • I suppose the -j8 option is not available for already clone repos? – Czar Pino Apr 10 '18 at 07:43
  • 1
    If the repos is not public and a token/authentication is needed, this cmd will ask you to type in again and again the access token/ or user_password for every submodule. Is there any workaround? – weefwefwqg3 Sep 10 '18 at 18:17
  • 27
    Also to make clear (since I wanted to know and couldn't find an answer except by looking at the source), the git clone `--recursive` and `--recurse-submodules` options behave identically. They result in a call to the same function. – Michael Burr Nov 28 '18 at 00:52
  • 4
    Why isn't recursive cloning the default option? – Minh Nghĩa Jul 22 '20 at 09:16
  • what happens if you do them in reverse? – Charlie Parker Jan 03 '23 at 20:22
  • why is your answer better than `git clone --recurse-submodules --remote-submodules `? – Charlie Parker Jan 03 '23 at 23:18
619

You have to do two things before a submodule will be filled:

git submodule init 
git submodule update
LiraNuna
  • 64,916
  • 15
  • 117
  • 140
  • 11
    I was afraid of that... it doesn't make any sense since you're checking out a partial project in that case. I understand that the submodule updates aren't automatic, but why isn't the bound version automatically checked out?? Is there any way to force it? I have a project with 3-levels of submodules and it seems absurd to have to traverse that far just to do a checkout. –  Sep 26 '10 at 07:26
  • 13
    Please read the `git-submodule(1)` man page (http://www.kernel.org/pub/software/scm/git/docs/git-submodule.html). You'll find out that `git submodule update` supports a nice parameter called `--recursive`. – joschi Sep 26 '10 at 07:30
  • 127
    Why not just do both of them in one command? `git submodule update --init` (Also see [my answer](http://stackoverflow.com/questions/3796927/git-clone-submodule/4438292#4438292)). – Mathias Bynens Dec 15 '10 at 13:04
  • 11
    I think its better to answer the question with these two commands. Its explains better how to accomplish the task. – schmijos Apr 17 '13 at 18:20
  • 6
    @MathiasBynens A machine that I just logged into only has git 1.5.5.6, which apparently does not support the shortened instruction, but does support it as two commands. – Jack Poulson Jul 24 '13 at 18:40
  • @MathiasBynens I think you misunderstood. ``git submodule update --init`` must be expanded to ``git submodule init; git submodule update`` in 1.5.5.6. – Jack Poulson Jul 25 '13 at 16:16
  • After I do this, then if I cd into the submodule directory and to `git pull` it says its head is detached and I can't do `git pull` in it to update, any way to fix this? – Noitidart Aug 18 '16 at 22:04
  • what happens if you do them in reverse? – Charlie Parker Jan 03 '23 at 20:22
346

Git 2.23 (Q3 2019): if you want to clone and update the submodules to their latest revision:

git clone --recurse-submodules --remote-submodules <repo-URL>

If you just want to clone them at their recorded SHA1:

git clone --recurse-submodules <repo-URL>

See below.

Note that Git 2.29 (Q4 2020) brings a significant optimization around submodule handling.

See commit a462bee (06 Sep 2020) by Orgad Shaneh (orgads).
(Merged by Junio C Hamano -- gitster -- in commit 2ce9d4e, 18 Sep 2020)

submodule: suppress checking for file name and ref ambiguity for object ids

Signed-off-by: Orgad Shaneh

The argv argument of collect_changed_submodules() contains only object ids (the objects references of all the refs).

Notify setup_revisions() that the input is not filenames by passing assume_dashdash, so it can avoid redundant stat for each ref.

Also suppress refname_ambiguity flag to avoid filesystem lookups for each object. Similar logic can be found in cat-file, pack-objects and more.

This change reduces the time for git fetch(man) in my repo from 25s to 6s.


Original answer 2010

As joschi mentions in the comments, git submodule now supports the --recursive option (Git1.6.5 and more).

If --recursive is specified, this command will recurse into the registered submodules, and update any nested submodules within.

See Working with git submodules recursively for the init part.
See git submodule explained for more.

With version 1.6.5 of git and later, you can do this automatically by cloning the super-project with the –-recursive option:

git clone --recursive git://github.com/mysociety/whatdotheyknow.git

Update 2016, with git 2.8: see "How to speed up / parallelize downloads of git submodules using git clone --recursive?"

You can initiate fetching the submodule using multiple threads, in parallel.
For instances:

git fetch --recurse-submodules -j2

Even better, with Git 2.23 (Q3 2019), you can clone and checkout the submodule to their tracking branch in one command!

See commit 4c69101 (19 May 2019) by Ben Avison (bavison).
(Merged by Junio C Hamano -- gitster -- in commit 9476094, 17 Jun 2019)

clone: add --remote-submodules flag

When using git clone --recurse-submodules there was previously no way to pass a --remote switch to the implicit git submodule update command for any use case where you want the submodules to be checked out on their remote-tracking branch rather than with the SHA-1 recorded in the superproject.

This patch rectifies this situation.
It actually passes --no-fetch to git submodule update as well on the grounds the submodule has only just been cloned, so fetching from the remote again only serves to slow things down.

That means:

--[no-]remote-submodules:

All submodules which are cloned will use the status of the submodule’s remote-tracking branch to update the submodule, rather than the superproject’s recorded SHA-1. Equivalent to passing --remote to git submodule update.


jhu notes in the comments:

If you want to use git clone --recurse-submodules --remote-submodules <repo-URL> to clone and update to the latest version, your submodules must either:

  1. have a branch master, assumed by git when running the above, or
  2. record a valid branch name in the .gitmodules of the cloned repo. > Otherwise you will get an error about a missing head, and cloning will fail.
    So if you have a submodule without branch master, say submodule sub with branch main, run in the root dir of the cloned repo git config -f .gitmodules submodule.sub.branch main and push your changes to the remote.
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • 12
    So it took Git 14 years to start adding proper support for submodules, huh. Thanks for the update! What if I already have a clone of the main repo without submodules and without a recorded SHA1, and I want to pull in the latest version of each submodule. Is it doable? – Violet Giraffe Sep 09 '19 at 09:48
  • 2
    @VioletGiraffe If that cloned repository has submodules, it has "recorded SHA1". And a `git submodule update --init --recursive --remote` should update them to the latest commit of their respective branch. (ex: https://stackoverflow.com/a/56981834/6309) – VonC Sep 09 '19 at 10:14
  • 1
    Let me clarify with an example: I have a template project on Github that uses submodules, and I even commited specific revisions of the submodules into this template repo. But when I create a new project out of this repo, none of the commands you listed (neither `clone --recurse-submodules --remote-submodules` nor `submodule update --init --recursive --remote`) let me actually fetch the subrepos. All I get is a .gitmodules file, and I couldn't find any way to init the subrepos other than manually cloning them one by one. I'd like to at least have a script to do it with `submodule foreach`... – Violet Giraffe Sep 10 '19 at 05:12
  • If you know a solution, I'd ask a separate question that you could answer. Here's the test repo that I can't find any way to init other than by hand: https://github.com/VioletGiraffe/TEST – Violet Giraffe Sep 10 '19 at 05:13
  • 1
    @VioletGiraffe That is because you have added and committed the .gitmodules but not the gitlink (https://stackoverflow.com/a/16581096/6309, special entries in the index: https://stackoverflow.com/a/19354410/6309) Here is a repository which does have the proper gitlink registered: https://github.com/tiagomazzutti/antlr4dart – VonC Sep 10 '19 at 06:20
  • @VioletGiraffe Solution: clone your repo, delete the .gitmodules, add, and commit. Then add the submodules again: `git submodule add cpputils https://github.com/VioletGiraffe/cpputils.git` and `git submodule add cpp-template-utils https://github.com/VioletGiraffe/cpp-template-utils`. Add, commit and push. You will then see your submodules, and can clone back that repository *with* said submodules. – VonC Sep 10 '19 at 06:22
  • Right, I understand that, and it works for any "normal" repo you create from scratch. I think we're looking at a Github bug with regards to handling subrepos in project templates specifically, because the original template repo has gitlinks as well, but not the new project created from this template. P. S. Thanks for the answers! – Violet Giraffe Sep 10 '19 at 07:01
  • @VioletGiraffe You could still update that project template with the process I mentioned. But yes, that looks like a bug on GitHub side. That would be interesting to illustrate/detail that in a separate question. I will answer it when I go back from work. – VonC Sep 10 '19 at 08:35
  • What exactly does `--remote-submodules`? Does it pull for each submodule the latest commit, and then update the reference in the superproject? – Aviv Cohn Jul 20 '20 at 07:02
  • @AvivCohn It does pull for each submodule the latest commit of their associated branch (`master/main` by default). To my understanding, you still need to add and commit in the parent repo, in order to record the new submodules tree SHA. – VonC Jul 20 '20 at 10:38
  • isn't your git clone cmd missing the repo url? – Charlie Parker Jan 03 '23 at 23:04
  • 1
    If you want to use `git clone --recurse-submodules --remote-submodules ` to clone and update to the latest version, your submodules *must* either 1. have a branch `master`, *assumed* by git when running the above, or 2. record a valid branch name in the `.gitmodules` of the cloned repo. Otherwise you will get an error about a missing head, and cloning will fail. So if you have a submodule without branch `master`, say submodule `sub` with branch `main`, run in the root dir of the cloned repo `git config -f .gitmodules submodule.sub.branch main ` and push your changes to the remote. – jhu Aug 20 '23 at 05:23
  • @jhu Good point, thank you for this feedback. I have included your comment in the answer for more visibility. – VonC Aug 20 '23 at 12:21
165

You can use this command to clone your repo with all the submodules:

git clone --recursive YOUR-GIT-REPO-URL

Or if you have already cloned the project, you can use:

git submodule init
git submodule update
wovano
  • 4,543
  • 5
  • 22
  • 49
Javier C.
  • 7,859
  • 5
  • 41
  • 53
  • On git version 2.24.3 the above command gives me the error: error: Server does not allow request for unadvertised object e635630d55682951eb2da35630d5da15b6cc Fetched in submodule path 'ui-library', but it did not contain e635630d55682951eb2da35630d5da15b6cc. Direct fetching of that commit failed. – BertC Apr 07 '21 at 10:54
  • ran the second option, nothing happens – Pedro77 Nov 12 '22 at 14:38
  • what happens if you do them in reverse? – Charlie Parker Jan 03 '23 at 20:23
118

[Quick Answer]

After cloning the parent repo (including some submodule repos), do the following:

git submodule update --init --recursive
Benyamin Jafari
  • 27,880
  • 26
  • 135
  • 150
47

Use this command to clone repo with all submodules

git clone --recurse-submodules git@gitlab.staging-host.com:yourproject

To update code for all submodules

git submodule update --recursive --remote
Antier Solutions
  • 1,326
  • 7
  • 10
39

If your submodule was added in a branch be sure to include it in your clone command...

git clone -b <branch_name> --recursive <remote> <directory>
Mars Redwyne
  • 1,267
  • 13
  • 8
  • This was more like what I was looking for... but the submodules list their branch as 'detached'. :( – AceFunk Jul 25 '19 at 13:25
34

Try this:

git clone --recurse-submodules

It automatically pulls in the submodule data assuming you have already added the submodules to the parent project.

nweiler
  • 1,140
  • 1
  • 13
  • 23
  • 41
    Note that `--recurse-submodules` and `--recursive` are [equivalent aliases](https://www.kernel.org/pub/software/scm/git/docs/git-clone.html). – Joel Purra Jan 16 '13 at 18:57
  • 1
    @SuperUberDuper in that case you can do `git submodule update --init --recursive` as explained in [this answer](https://stackoverflow.com/a/4438292/854076) – Enrico Mar 19 '16 at 15:43
28

I think you can go with 3 steps:

git clone
git submodule init
git submodule update
NG_
  • 6,895
  • 7
  • 45
  • 67
muhammad ali e
  • 655
  • 6
  • 8
24

You can use the --recursive flag when cloning a repository. This parameter forces git to clone all defined submodules in the repository.

git clone --recursive git@repo.org:your_repo.git

After cloning, sometimes submodules branches may be changed, so run this command after it:

git submodule foreach "git checkout master"
Vincent Beltman
  • 2,064
  • 13
  • 27
Ahmad Azimi
  • 643
  • 1
  • 7
  • 12
  • + 1 foreach but not enough details. see this for a more detailed answer: https://stackoverflow.com/questions/74988223/why-do-i-need-to-add-the-remote-to-gits-submodule-when-i-specify-the-branch/74999399#74999399 – Charlie Parker Jan 03 '23 at 23:03
  • The `checkout` done here switches from a snapshot (detached head) to a working branch. And, say you have some submodules where the branch you want to work with is `master`, and some submodules where the branch you want to work with is `main`. (This is quite common now, since `master` used to be the default, but now it is typically `main`.) Then you can check out a working branch of *all* submodules by running `git submodule foreach --recursive "git checkout master || git checkout main" ` – jhu Aug 20 '23 at 05:27
22

late answer

// git CLONE INCLUDE-SUBMODULES ADDRESS DESTINATION-DIRECTORY
git clone --recursive https://USERNAME@bitbucket.org/USERNAME/REPO.git DESTINATION_DIR

As I just spent a whole hour fiddling around with a friend: Even if you have Admin rights on BitBucket, always clone the ORIGINAL repository and use the password of the one who owns the repo. Annoying to find out that you ran into this minetrap :P

kaiser
  • 21,817
  • 17
  • 90
  • 110
  • That's exactly what I'm dealing with. So, are you saying that anyone who needs to develop on a bitbucket repository that has submodules must use the repository creator's credentials? Blech. – jsleuth May 29 '13 at 19:28
  • @jsleuth Seems so - it sucks BIG TIME... and I know it. – kaiser May 29 '13 at 20:35
  • That sounds like a bug. Did you report it to Bitbucket? – Mathias Bynens Sep 05 '14 at 19:51
  • @MathiasBynens Did you stumble upon this issue? It's one and a half years later and I actually don't know if this is still the case. – kaiser Sep 05 '14 at 21:31
  • @kaiser No, I just stumbled upon your answer and was surprised. – Mathias Bynens Sep 07 '14 at 12:23
  • This is the case only if you use HTTPS and the user is in the URL? The "right" way is to use the SSH auth with a private/public key pair, so the password would not be requested. – Lucas Freitas Jul 05 '16 at 07:55
  • 1
    @kaiser This answer should really be deleted – Treffynnon Feb 13 '17 at 02:46
  • 5
    It doesn't descriptively answer the OPs question, but details an unrelated bug in Bitbucket; which, incidentally, could just be shortened to "use SSH key authentication". – Treffynnon Feb 14 '17 at 03:44
  • 1
    Still not resolved on Bitbucked. I'm using relative paths to submodules and as a trick I have to perform following steps: `- git remote set-url origin git@bitbucket.org:namespace/main-repo.git` `- git submodule update --init --recursive` – niziak Feb 10 '22 at 13:22
21

Try this for including submodules in git repository.

git clone -b <branch_name> --recursive <remote> <directory>

or

git clone --recurse-submodules
4b0
  • 21,981
  • 30
  • 95
  • 142
radhey shyam
  • 759
  • 6
  • 9
18

Just do these in your project directory.

$ git submodule init
$ git submodule update
Rajitha Udayanga
  • 1,559
  • 11
  • 21
12

Submodules parallel fetch aims at reducing the time required to fetch a repositories and all of its related submodules by enabling the fetching of multiple repositories at once. This can be accomplished by using the new --jobs option, e.g.:

git fetch --recurse-submodules --jobs=4

According to Git team, this can substantially speed up updating repositories that contain many submodules. When using --recurse-submodules without the new --jobs option, Git will fetch submodules one by one.

Source: http://www.infoq.com/news/2016/03/git28-released

Mafii
  • 7,227
  • 1
  • 35
  • 55
Long Nguyen
  • 145
  • 1
  • 3
10

If it is a new project simply you can do like this :

$ git clone --recurse-submodules https://github.com/chaconinc/YourProjectName 

If it is already installed than :

$ cd YourProjectName (for the cases you are not at right directory) 
$ git submodule init
$ git submodule update
nzrytmn
  • 6,193
  • 1
  • 41
  • 38
10

I had the same problem for a GitHub repository. My account was missing SSH Key. The process is

  1. Generate SSH Key
  2. Adding a new SSH key to your GitHub account

Then, you can clone the repository with submodules (git clone --recursive YOUR-GIT-REPO-URL)

or

Run git submodule init and git submodule update to fetch submodules in already cloned repository.

FatalError
  • 922
  • 12
  • 31
  • Yes, that is `Permission denied (publickey). fatal: Could not read from remote repository.` error – Ender Nov 14 '19 at 09:34
7

Try this.

git clone -b <branch_name> --recursive <remote> <directory>

If you have added the submodule in a branch make sure that you add it to the clone command.

4b0
  • 21,981
  • 30
  • 95
  • 142
Himeshika96
  • 79
  • 1
  • 1
1

I recommend:

# - git submodule init initializes your local configuration file to track the submodules your repository uses, it just sets up the configuration so that you can use the git submodule update command to clone and update the submodules.
git submodule init
# - git submodule update --init initializes your local configuration file and clones the submodules for you, using the commit specified in the main repository.
#   note, command bellow will not pull the right branch -- even if it's in your .gitmodules file, for that you need remote. Likely because it looks at the origin (pointer to remote) in github for the available branches.
#   note, bellow pulls the submodules if you didn't specify them when cloning parent project, ref: https://youtu.be/wTGIDDg0tK8?t=119
git submodule update --init

if you have a specific branch for your submodules then change it to:

# - git submodule init initializes your local configuration file to track the submodules your repository uses, it just sets up the configuration so that you can use the git submodule update command to clone and update the submodules.
git submodule init

# - The --remote option tells Git to update the submodule to the commit specified in the upstream repository, rather than the commit specified in the main repository.
#git submodule update --init --remote
git submodule update --init --recursive --remote meta-dataset

For a full example that was testing:

# decided against this because it seems complicated

# - note to clone uutils with its submodule do (cmd not tested):
cd $HOME
git clone --recurse-submodules git@github.com:brando90/ultimate-utils.git

# - git submodules
cd $HOME/diversity-for-predictive-success-of-meta-learning

# - in case it's needed if the submodules bellow have branches your local project doesn't know about from the submodules upstream
git fetch

# -- first repo
# - adds the repo to the .gitmodule & clones the repo
git submodule add -f -b hdb --name meta-dataset git@github.com:brando90/meta-dataset.git meta-dataset/

# - ref for init then update: https://stackoverflow.com/questions/3796927/how-do-i-git-clone-a-repo-including-its-submodules/3796947#3796947
#git submodule init
#git submodule update

# - git submodule init initializes your local configuration file to track the submodules your repository uses, it just sets up the configuration so that you can use the git submodule update command to clone and update the submodules.
git submodule init
# - git submodule update --init initializes your local configuration file and clones the submodules for you, using the commit specified in the main repository.
#   note, command bellow will not pull the right branch -- even if it's in your .gitmodules file, for that you need remote. Likely because it looks at the origin (pointer to remote) in github for the available branches.
#   note, bellow pulls the submodules if you didn't specify them when cloning parent project, ref: https://youtu.be/wTGIDDg0tK8?t=119
git submodule update --init
# - The --remote option tells Git to update the submodule to the commit specified in the upstream repository, rather than the commit specified in the main repository.
#git submodule update --init --remote
git submodule update --init --recursive --remote meta-dataset

# - check we are using the right branch https://stackoverflow.com/questions/74998463/why-does-git-submodule-status-not-match-the-output-of-git-branch-of-my-submodule
git submodule status
cd meta-dataset
git branch  # should show hdb
cd ..

# pip install -r $HOME/meta-dataset/requirements.txt

# -- 2nd repo, simplified commands from above
git submodule add -f -b hdb --name pytorch-meta-dataset git@github.com:brando90/pytorch-meta-dataset.git pytorch-meta-dataset/

git submodule init
git submodule update --init --recursive --remote meta-dataset

# - check it's in specified branch
git submodule status
cd pytorch-meta-dataset
git branch  # should show hdb
cd ..

# pip install -r $HOME/pytorch-meta-dataset/requirements.txt
Charlie Parker
  • 5,884
  • 57
  • 198
  • 323
1

If you want to tell your git client to do all actions with --recurse-submodules then you can set this to your git config:

git config submodule.recurse true

See: https://www.git-scm.com/docs/git-config#Documentation/git-config.txt-submodulerecurse

wovano
  • 4,543
  • 5
  • 22
  • 49
  • Unless this is for a newer git than I have, this does not appear to work (even with `--global`). – Compholio Mar 14 '23 at 16:27
  • @Compholio `submodule.recurse` is documented in `man git config` in my `git version 2.39.2` – Guildenstern Mar 18 '23 at 17:51
  • Yeah, the documentation that you have linked to says "clone and ls-files are not supported." So, I do not believe that config option works to do this (it works for other things, just not clone). – Compholio Mar 20 '23 at 15:39
0
git submodule init
git submodule update

or maybe:

git stash -u
git pull origin master
git stash p
wovano
  • 4,543
  • 5
  • 22
  • 49
  • 2
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Apr 13 '22 at 16:05
0

If the submodules are private submodules you can use credential store so that it also clones its private submodules recursively.

USER=${GITHUB_ACTOR}
TOKEN=${{ secrets.JEKYLL_GITHUB_TOKEN }}

git config --global credential.helper store
echo "https://${USER}:${TOKEN}@github.com" > ~/.git-credentials

git clone --recurse-submodules -j8 git://github.com/foo/bar.git
cd bar

I use it to clone my submodules where the private one is in 5th level deep. Please allow me to show you how it goes in action:

enter image description here

eQ19
  • 9,880
  • 3
  • 65
  • 77
0

You can copy clone url from github.

Then use:

git clone --recursive <url of clone copied above>
wovano
  • 4,543
  • 5
  • 22
  • 49
  • 1
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jan 18 '23 at 15:28
-2

When you clone a repository with submodules, you need to initialize and update the submodules separately. Here's how you can pull a repository with submodules and update them:

  1. Open a terminal on your Ubuntu system.

  2. Change to the directory where your repository is located:

    cd /path/to/repository
    
  3. Use the following command to clone the repository and its submodules:

    git clone --recurse-submodules <repository_url>
    

    Replace <repository_url> with the actual URL of your repository.

    The --recurse-submodules flag ensures that the submodules are also cloned along with the main repository.

  4. If you have already cloned the repository without initializing and updating the submodules, navigate to the repository directory and initialize the submodules:

    cd /path/to/repository
    git submodule init
    
  5. Update the submodules to retrieve the latest changes from their respective repositories:

    git submodule update
    

    If you want to update the submodules recursively (i.e., update submodules within submodules), you can add the --recursive flag:

    git submodule update --recursive
    

    This command will fetch the latest commits for each submodule.

After completing these steps, your repository and its submodules should be up to date. You can now use the files from the submodules as expected.

Ankit Kumar
  • 403
  • 3
  • 8
-4

How do I "git clone" a repo, including its submodules?

Is there any way to specify this behavior as default in your git repository, so that less-informed cloners will automatically get an initialized submodule?

(I made it as a standalone answer because I think this IS important in the git)

Sadly, but there is plenty cases where that less-informed cloners will ANYWAY miss the externals:

Note:
Such projects as vcstool, git-subrepo and gil are using their own way to store external dependencies list and supersedes the clone operation, so the end repository by default won't has the .submodules file at all.
Here is some clarification of raised problem in the git internals:
nested submodules detection w/o .gitmodules file :
https://lore.kernel.org/git/1716310675.20230122233403@inbox.ru

As a backup, inability to make a bare clone (--bare) + full recursive (--recursive) to mirror (--mirror) all references (not just all externals):
Clone git repository with --recursive and --bare

So with the git there is no a builtin way to take the external dependencies absolutely completely. Which means in turn there is no a single way to backup any random git repository absolutely completely.

Andry
  • 2,273
  • 29
  • 28
  • 1
    Downvoted because you didn’t answer the question. The question has nothing to do with backup whatsoever. – Alex Aug 01 '23 at 00:21
  • @Alex I've answered to the most upvoted comment about "less informed" users. Not to the main question. And the backup IS a full clone. – Andry Aug 02 '23 at 16:11