2

Have just created a new repository with two submodules. It works fine on local but when pushing to the production the submodules are not there. Each submodule root directory is present, but no files inside

git version 2.27.0

Create repo on production

git --bare init
cd hooks && touch post-receive && chmod +x post-receive

cat hooks/post-receive
#/bin/sh
git --work-tree=/var/www/repo --git-dir=/var/git_repos/repo.git checkout -f

Pushing to production from dev/local

git push production master

git status tells that everything is up to date. Even if it is called in the submodule folder. Have tried to delete the submodule folder on the local repository, commit/push to repository. Then git push production master and the root submodule folder was deleted on the production server. Then trying to add it again

# git submodule add git@github.com:alias/repo_name.git php/repo/repo_name
Cloning into '/var/www/project/php/repo/repo_name'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 19 (delta 8), reused 19 (delta 8), pack-reused 0
Receiving objects: 100% (19/19), 8.62 KiB | 8.62 MiB/s, done.
Resolving deltas: 100% (8/8), done.

Then commiting to repository and then git push production master . The same problem. In production the root folders of the submodules are present but no files in the directories.

When I browse the repository on github, the submodules are correctly linked.

Everything works in the local repository

enter image description here

clarkk
  • 27,151
  • 72
  • 200
  • 340
  • 1
    Once you add a submodule, it is not sufficient to clone. `git submodule init` and `git submodule update` are also needed. – Daemon Painter Nov 26 '20 at 12:25
  • Does this answer your question? [Empty Git submodule folder when repo cloned](https://stackoverflow.com/questions/11358082/empty-git-submodule-folder-when-repo-cloned) – Daemon Painter Nov 26 '20 at 12:40
  • I'm assuming that production repo is not a bare repository here, and that you are inspecting it via a clone (by the looks of it). If not, please put more details. – Daemon Painter Nov 26 '20 at 12:41
  • I am not cloning but pushing to production – clarkk Nov 26 '20 at 13:20
  • How to you inspect the content of Production repository? – Daemon Painter Nov 26 '20 at 13:27
  • I'm new to git (started yesterday). What do you mean? – clarkk Nov 26 '20 at 13:33
  • When I browse the parent repository on github both subrepositories (submodules) are correctly linked – clarkk Nov 26 '20 at 13:34
  • Assuming you are checking the right branch on GitHub, the folder cannot be empty: Git doesn't show empty folders. So either you are describing a link (GitHub uses a different icon for submodules, maybe you could include a screenshot) or I've lost you entirely. – Daemon Painter Nov 26 '20 at 13:44
  • screen dump added – clarkk Nov 26 '20 at 13:49
  • The submodule folders are not empty on github (or in local repo), but when pushing to production they are – clarkk Nov 26 '20 at 14:15
  • so the screenshot is not from the production environment? – Daemon Painter Nov 26 '20 at 14:19
  • no.. this is on github.. in production the submodules are present but they are empty – clarkk Nov 26 '20 at 14:22
  • have now tried to do `git submodule init` and `git submodule update` in the root of each submodule.. same problem.. have even made a new commit and push to the parent repository and then `git push production master` – clarkk Nov 26 '20 at 14:47

1 Answers1

0

The submodule folders are not empty on github (or in local repo), but when pushing to production they are

That is expected: a submodule is composed of:

  • an URL registered in the .gitmodules
  • a gitlink, a special entry in the index which represents the SHA1 of the root tree of the submodule repository.

When you clone a remote repository with submodules (using the git clone --recurse-submodules option, those submodule repositories are cloned themselves, and checked out at the SHA1 represented by that gitlink.

"Pushing to production" is tricky because you are not supposed to push to a checked out (non-bare) repository (even thou you can technically do it).

It is better to push to a bare repository on production, which will use a post-receive hook to restore the files (including the submodule contents, using git restore --recurse-submodules)

After discussion:

  1. You need for the production server to have a public SSH key registered to your account, since the submodules are registered in the .gitmodules with an SSH URL.

  2. The post-receive script becomes:

    #!/bin/bash
cd /var/www/repo
git --git-dir=/var/git_repos/repo.git --work-tree=. restore --recurse-submodules :/

With git submodule, you cannot set the working tree apparently

That means, if submodules are involved, you current folder must be the working tree.

Then you can do commands like git checkout/git restore with --recurse-submodules, or git submodule update --init, even with a different git-dir (--git-dir=/path/to/bare/repo.git).

The submodules will be updated because your current working tree is the right one.

If you are anywhere than /var/www/repo, even if you specify git --work-tree=/var/www/repo ..., none of the aforementioned commands git checkout/git restore with --recurse-submodules, or git submodule update --init will work for the submodules of your repo.
That is: Your /var/www/repo files will be restored, but the submodules will remain empty.

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • So what would be best practice in this case? – clarkk Nov 27 '20 at 20:59
  • is this command not the right one to add a submodule? `git submodule add git@github.com:alias/repo_name.git php/repo/repo_name` – clarkk Nov 27 '20 at 21:03
  • @clarkk Either having a bare repo on prod to which you can push directly (with its associated post-receive hook), or a non-bare repo on prod, and an on-demand process which would do a `git pull --recurse-submodules` from GitHub (meaning you push to GitHub, and pull from prod) – VonC Nov 27 '20 at 21:04
  • I created the repo on production like this: `git --bare init` and have already a post-receive hook with this: `git --work-tree=/var/www/repo --git-dir=/var/git_repos/repo.git checkout -f` ... So I just have to add a new line in the post-receive with: `git pull --recurse-submodules` ? – clarkk Nov 27 '20 at 21:11
  • @clarkk No need for pull, since you are pushing to it. All you need to do is use `git checkout -f --recurse-submodules` (or, if your Git is modern enough: `git restore -f --recurse-submodules`) – VonC Nov 27 '20 at 21:12
  • Ok, so I just have to append the command in the post-receive hook with `--recurse-submodules` so the final command will be: `git --work-tree=/var/www/repo --git-dir=/var/git_repos/repo.git checkout -f --recurse-submodules` ? – clarkk Nov 27 '20 at 21:15
  • @clarkk Yes that is the idea, provided you are pushing directly to that bare repo on prod. – VonC Nov 27 '20 at 21:16
  • I will always be pushing to production from a remote client (the local dev) with `git push production master` – clarkk Nov 27 '20 at 21:17
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/225230/discussion-between-vonc-and-clarkk). – VonC Nov 27 '20 at 21:18
  • I have just upgraded to `git version 2.27.0` both on dev and production server. In my post-receive hook I have one command: `git --work-tree=/var/www/repo --git-dir=/var/git_repos/repo.git restore --recurse-submodules` The submodule folders are still empty after pushing to production – clarkk Nov 28 '20 at 12:11
  • Sorry to bother you again. I was only testing the commands in the post-receive directly on production, but when running `git push production master` from remote I get an error `remote: fatal: failed to unpack tree object d09a53c53a7bd222df4869e66b9e45acee45e094 remote: error: Submodule 'php/repo/utils' could not be updated.` – clarkk Nov 28 '20 at 17:29