How to take backup of git from one server to another

Question

I have gitolite setup on one one of my server and it has lots of git repositories. So i decided to take a back up of all these and found this about git bundle. And based on it created the below script

#!/bin/bash
projects_dir=/home/bean/git/
backup_base_dir=/home/bean/gitbackup/

test -d ${backup_base_dir} || mkdir -p ${backup_base_dir}
pushd $projects_dir

for repo in `find $projects_dir -type d -name .git`; do
    cd $repo/..
    this_repo=`basename $PWD`
    git bundle create ${backup_base_dir}/${this_repo}.bundle --all
    cd -
done
popd

cd $backup_base_dir
rsync  -r . username@ip_address:/home/bean1/git_2nd_backup

In the above script 1st i am taking a backup of the repository in the same machine but in different folder as mentioned in the script backup_base_dir=/home/bean/gitbackup/ and then using rsync taking the backup in another machine/server. So my question is, is there any way to avoid the backing up in the same machine and i can directly backup to another server/machine. Just wanted to eliminate the backing up in same machine and wanted to backup directly to server machine.

Both the machines/servers are Ubuntu 16

Answer 1

For GitHub, this works:

for repo in $(/usr/local/bin/curl -s -u <username>:<api-key> "https://api.github.com/user/repos?per_page=100&type=owner" | 
    /usr/local/bin/jq -r '.[] | .ssh_url')
do
   /usr/local/bin/git clone --mirror ${repo}
done

To follow the same approach you will need to expose as-as service a list of all your repositories, here I created something small in go that hopefully could help you as a start point:

https://gist.github.com/nbari/c98225144dcdd8c3c1466f6af733c73a

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "os"
    "os/exec"
    "path/filepath"

    "github.com/nbari/violetear"
)

type Repos struct {
    Remotes []string
}

func findRemotes(w http.ResponseWriter, r *http.Request) {
    files, _ := filepath.Glob("*")
    repos := &Repos{}
    for _, f := range files {
        fi, err := os.Stat(f)
        if err != nil {
            fmt.Println(err)
            continue
        }
        if fi.Mode().IsDir() {
            out, err := exec.Command("git",
                fmt.Sprintf("--git-dir=%s/.git", f),
                "remote",
                "get-url",
                "--push",
                "--all",
                "origin").Output()
            if err != nil {
                log.Println(err)
                continue
            }
            if len(out) > 0 {
                repos.Remotes = append(repos.Remotes, strings.TrimSpace(fmt.Sprintf("%s", out))) 
            }
        }
    }
    if err := json.NewEncoder(w).Encode(repos); err != nil {
        log.Println(err)
    }
}

func main() {
    router := violetear.New()
    router.HandleFunc("*", findRemotes)
    log.Fatal(http.ListenAndServe(":8080", router))
}

Basically, will scan all directories from where you run the code on your server and try to find the git remotes returning in a JSON format a list of all remotes found.

From your backup server later you could just use the same approach used for Github and backup without duplication.

In this case, you could use it like this:

for repo in $(curl -s your-server:8080 | jq -r '.[][]')
do
   /usr/local/bin/git clone --mirror ${repo}
done