0

Sometimes, one may need to execute the following commands to clone a repo to local. The repo is around 100GB. So is there a way/how to setup to 'clone' the repo in the remote server 'examplecode.org' to a local server so that it didn't need to fetch the codebase remotely?

Originally, the commands to execute are:

$repo init -u git://examplecode.org/platform/manifest.git -b release -m ${MyTag} --repo-url=git://examplecode.org/tools/repo.git --repo-branch=my-branch-stable
$repo sync -f -j${CPUS}

Let's 192.168.1.100 be the server in local network, ideally, after the setup, the commands would be like:

$repo init -u git://192.168.1.100/platform/manifest.git -b release -m ${MyTag} --repo-url=git://192.168.1.100/tools/repo.git --repo-branch=my-branch-stable
$repo sync -f -j${CPUS}

repo means git-repo

Many thanks

Laurence W
  • 93
  • 6
  • How do you imagine that the repository is stored locally without downloading it? – mkrieger1 Dec 17 '19 at 00:26
  • And what are those commands? For `init`, did you mean `git init`? And what is `sync` - there is no Git command with that name. – mkrieger1 Dec 17 '19 at 00:29
  • Hi @mkrieger, 1. ideally, it will download the code base once to a local server, then others can clone a working one from the local server rather than from the remote one, "examplecode.org" 2. `repo` mean [git-repo](https://gerrit.googlesource.com/git-repo) – Laurence W Dec 17 '19 at 00:40
  • I don't know that tool, but of course it is possible to do what you just described with Git. Did you try it? What exactly was the problem? – mkrieger1 Dec 17 '19 at 00:42
  • Similar to 'git --bare', to clone a local replicate of a remote repository to share to others. However, it is 'repo' command. – Laurence W Dec 17 '19 at 00:46
  • Yes. So did you try to run these commands you've shown in the question? What happened then? – mkrieger1 Dec 17 '19 at 00:48
  • The keypoint is how to setup the local replicate. – Laurence W Dec 17 '19 at 00:51

2 Answers2

0

Your repo is 100 GB? If that's not a typo, you might consider ways to reduce that size.

A good way to start might be by looking for logical ways to split the existing code (and probably a lot of non-code...) across multiple repositories. Is the code base sufficiently modular, such that you could move different modules into their own repository?

And speaking of non-code, are there a lot of large binary files being kept in the repo? Maybe those could be taken out of the version control system and moved elsewhere,like a LAN drive or a sharepoint site. A lot of the benefits of version control are lost with binary files and they just end up taking a ton of space.

Last, you might want to run git-gc to clean/optimize the repository. This is especially true if you are self-hosting a centralized "master" server (rather than using Github or something similar), since that may not be happening automatically at regular intervals.

Edit:
I realize this doesn't directly answer the stated question. But it seems OP may have some other areas to explore that would make the sought after solution less urgent.

Edit 2:
Based on the comments, it sounds like there is some confusion about what git does versus what repo (the Android wrapper for git) gives you on top of git. First, and most importantly, you can freely switch between repo and git commands. Using one does not bar you from using the other. The repo script is just a wrapper that simplifies some operations that are common when developing for Android, but which are cumbersome to do manually with git. Outside of those operations, most of the repository manipulation will probably be done using the standard git commands.

Regarding the original question, what you are describing is the git clone command:

NAME
git-clone - Clone a repository into a new directory

SYNOPSIS

git clone [--template=<template_directory>]
          [-l] [-s] [--no-hardlinks] [-q] [-n] [--bare] [--mirror]
          [-o <name>] [-b <name>] [-u <upload-pack>] [--reference <repository>]
          [--dissociate] [--separate-git-dir <git dir>]
          [--depth <depth>] [--[no-]single-branch] [--no-tags]
          [--recurse-submodules[=<pathspec>]] [--[no-]shallow-submodules]
          [--jobs <n>] [--] <repository> [<directory>]

DESCRIPTION
Clones a repository into a newly created directory, creates remote-tracking branches for each branch in the cloned repository (visible using git branch -r), and creates and checks out an initial branch that is forked from the cloned repository's currently active branch.

After the clone, a plain git fetch without arguments will update all the remote-tracking branches, and a git pull without arguments will in addition merge the remote master branch into the current master branch [...]

Since you specifically mentioned wanting to have the code base on the local network, you would first clone the repository and then pull all branches:

git clone [remote URL]
git pull --all

Once that is done, you can use any repo and regular git commands to manage your local copy of the repository.

Z4-tier
  • 7,287
  • 3
  • 26
  • 42
  • After `repo sync`, the codebase is >=100GB is possible, for instance, Android's codebase. However, the size is not a issue. One may still consider setup a local copy to share in a office / network is not stable and so on. – Laurence W Dec 17 '19 at 00:44
  • If you're really just trying to make a local copy of a remote repository, why can't you just use `git clone [remote url]`??? Using `repo` doesn't preclude you from using the typical `git` workflows, and it sounds like what you're describing is really just `git clone`. After you clone the repository you should still be able to work with it using `repo` commands as well. – Z4-tier Dec 20 '19 at 01:06
0

Found 2 ways to do it:

  1. Using repo's --mirror option to create a local mirror, then use --reference to checkout by referencing the mirror

  2. Using grokmirror, pls ref https://git.kernel.org/pub/scm/utils/grokmirror/grokmirror.git/tree/README.rst

Laurence W
  • 93
  • 6