20

I have a website directory versioned with git. I use submodules for required libraries like Twitter Bootstrap, colorbox and lessjs because I should not track the sourcecode but only the version of their code I use.

Now I want to deploy the project and therefore I need the submodule code too. By using git archive I don't get the source files/code from the submodules.

The following three approaches TRY to achieve what I want but dont't work: 1st approach

#!/bin/sh

export revision="$1"

export GIT_INDEX_FILE=".git/tmpindex"
rm -f "$GIT_INDEX_FILE"

git read-tree $revision

export up="$(pwd)"

read_one_level () {
        export GIT_ALTERNATE_OBJECT_DIRECTORIES="$GIT_ALTERNATE_OBJECT_DIRECTORIES":$(
            git submodule foreach 'echo "$up/$path/.git/objects"' |
            grep -E -v '^(Entering|No submodule mapping found)' |
            tr '\n' : |
            sed 's/:$//'
        )

        git submodule foreach '
                cd "$up"
                subcommit=$(git rev-parse :"$path")
                git rm --cached "$path"
                git read-tree -i --prefix="$path/" $subcommit
        ' >/dev/null
}

while git ls-files -s | grep -q ^160000; do
    read_one_level
done

git archive --format=tar $(git write-tree)

rm -f "$GIT_INDEX_FILE"

Thomas Rast in http://git.661346.n2.nabble.com/Running-git-archive-recursively-over-submodules-td4577012.html

This gives me errors both on Windows as in Linux that there are no objects files found.

2nd approach https://github.com/meitar/git-archive-all.sh

Complains about mktemp not found on Windows. And correcting calls to git archive (from git-archive) does not give submodules in the tar ...:(

3rd approach https://github.com/Kentzo/git-archive-all

Is outdated from my point of view by not being compatible to latest python 3.3 and still not fully working by using 2.7 because samefile complains.

So my question is now: Is there any recent way/approach to deal with exporting/archive a git project including submodules?

Or should I check subtrees for this workflow?

Thanks in advance

kmindi
  • 4,524
  • 4
  • 31
  • 47
  • I'm the author of Kentzo/git-archive-all. It's python 3.3 compatible now. What is the other issue? – Kentzo May 13 '13 at 07:41

7 Answers7

9

I'm using the following code

git archive -o release.zip HEAD
git submodule --quiet foreach 'cd $toplevel; zip -ru release.zip $sm_path'

to create a complete archive of a git repository with all submodules.

If you want to be fancy you can even rewrite the zip comment via

echo -e "Repository:\n$(git rev-parse HEAD)\nSubmodule status:\n$(git submodule status)" | zip -u release.zip -z

All on windows using infozip.

t-b
  • 152
  • 1
  • 7
7

https://github.com/Kentzo/git-archive-all is now working under linux as @Kentzo stated.

In the cygwin environment (with git and python installed via cygwin installer) it works there too.

kmindi
  • 4,524
  • 4
  • 31
  • 47
  • git-archive-all is available at (https://pypi.org/project/git-archive-all/), install it with `pip install git-archive-all`... but caution: takes as output file only .tgz format. – Erich Kuester Dec 25 '20 at 17:32
  • Sorry, I just detected that my system (Fedora 33) had installed version 1.17 ... the latest version has no problems with tar, tar.gz, tar.xz and so on. – Erich Kuester Dec 27 '20 at 13:37
4

If you are a friend of KISS as me you could use the answer of @t-b BUT as I found this solution does not archive nested submodules. The following code will help

# archive main directory
$ git archive --format tar -o release.tar HEAD
# descend recursively and archive each submodule
$ git submodule --quiet foreach --recursive 'git archive --format tar --prefix=$displaypath/ -o submodule.tar HEAD'
# concatenate with main archive
$ TOPDIR=$(pwd) git submodule --quiet foreach --recursive 'cd $TOPDIR; tar --concatenate --file=release.tar $displaypath/submodule.tar; rm -fv $displaypath/submodule.tar'
$ gzip -9 release.tar

The result will be file result.tar.gz... instead of HEAD you could choose another commit, of course.

WofWca
  • 571
  • 3
  • 11
Erich Kuester
  • 460
  • 4
  • 11
2

We could do a quick bash script, where archive in tar, module and sub-modules, after we concatenate tars and compress, here it is an example with 2 sub modules:

#!/bin/bash

set -e

pwd=$(pwd)
date=$(date +%Y%m%d)
package=gitreponame
branch=master
name=tarbz2name
submodule1=/src/app/sub1/
submodule2=/src/sub2/

pushd ${package}
git checkout ${branch}
git pull
tag=$(git rev-list HEAD -n 1 | cut -c 1-7)
git archive --prefix="${name}/" --format=tar master > "$pwd"/${name}-${date}-${tag}.tar

git submodule update --init

cd ${submodule1}
git archive  --prefix="${name}${submodule1}" --format=tar master > "$pwd"/sb1.tar
cd -

cd ${submodule2}
git archive --prefix="${name}${submodule2}" --format=tar master > "$pwd"/sb2.tar
popd

tar -Af ${name}-${date}-${tag}.tar sb1.tar
tar -Af ${name}-${date}-${tag}.tar sb2.tar
bzip2 ${name}-${date}-${tag}.tar
rm sb1.tar sb2.tar
Sérgio
  • 6,966
  • 1
  • 48
  • 53
1

The 2nd approach: https://github.com/meitar/git-archive-all.sh deploys as a standalone Bash script and appears to be still-maintained. I found it to be an easy way to take a quick submodule inclusive snapshot of a project. It can even capture an earlier revision (sort of, see below):

git-archive-all.sh --tree-ish (tag) ## for example

A couple of problems that may or may not be deal breakers:

  • I ran into problems trying to use the --format option -- but the default, tar, generally worked.

  • The (very) big problem: It captured my submodule, but only in its current version -- not the specified version. Whoops!

Being a Bash script, this probably requires some kind of Linux compatibility layer (like Cygwin) to be run under Windows.

It seems like this should be a built-in capability of Git -- it's too complicated to piece together with scripting. I guess not many people use submodules or git-archive, so even fewer want to use both.

Brent Bradburn
  • 51,587
  • 17
  • 154
  • 173
  • 1
    The noted benefit of this approach was *it can even capture an earlier revision* (although it turned out to not really work). I don't see how to do that automatically with the @Kentzo solution either . The @Sergio solution appears to (sort of) do this, but I haven't yet tested it myself. I guess you have to do an actual checkout (and submodule update) with *all* of these solutions -- something that isn't required for a regular `git archive`. – Brent Bradburn Apr 27 '15 at 18:36
1

I have developed git-archive-with-submodules using bash. It also has the nice feature of allowing to create an archive including uncommitted changes.

Chris Maes
  • 35,025
  • 12
  • 111
  • 136
0

A simple solution that recursively calls git archive to zip up submodules.

The script should be placed in the Git repository's root directory (i.e. where the .git file is located).

The script hard-codes the following:

  • path to script
  • path to export directory.
#!/bin/bash
###
# Script name: archive_git_repos_as_zips.sh
# Purpose: Recursively produce zip archives of nested Git repositories.
###

# Path to this script
export SCRIPT='/PATH/TO/SCRIPT/archive_git_repos_as_zips.sh'

# Target directory for storing zip files
ZIP_DIR='/PATH/TO/git_zip_exports'

PWD=`pwd`
THIS=`basename $PWD`

# Zip file path
ZIP_FILE="$ZIP_DIR/${THIS}.zip

echo -e "\nCreating: $ZIP_FILE"
git archive -o $ZIP_FILE HEAD
git submodule --quiet foreach '$SCRIPT'

# end #
Adam99990000
  • 1
  • 2