0

I am trying get specific sub folders in a different repositories and would like to know how to run an automated script to get all desired subfolders downloaded to my local machine or pulled to my local git repo.

I have tried using the "sparse-checkout" on Git Bash however it does return the subfolders and their files as needed.

Any assistance on this will be appreciated.

Thanks!. Caleb

Caleb Adepoju
  • 115
  • 8
  • 1
    Is there a reason you can't just clone the repository and then work with the folders as desired (or copy them elsewhere)? – bk2204 Dec 03 '21 at 01:54
  • Tried something from https://stackoverflow.com/questions/600079/how-do-i-clone-a-subdirectory-only-of-a-git-repository ? Found in https://stackoverflow.com/search?q=%5Bgit%5D+clone+subfolder – phd Dec 03 '21 at 11:39
  • @bk2204 I am work on over 200 repos with total number of branches amounting to thousands across board. So I need to be able to use the CLI to specify each repos and down the folder/file needed – Caleb Adepoju Dec 03 '21 at 15:52

1 Answers1

2

In general, you can do this by cloning the repositories to your system with a shell script and then extracting the files from your system. If you'd rather download less data, you can use a partial clone (e.g., git clone --filter=blob:none) and then only the blobs you need will be pulled down on demand.

If you just need a branch or two from each repository and they're all from GitHub, you can use the GitHub REST API to download a tarball for the branch and repository you need. Note that you'll need to use a personal access token if you make more than 60 requests per hour.

If you have a partial clone, you can also use sparse checkout to filter only the files you want. However, you could also use git archive to produce a tarball, and if you have a tarball, either from git archive or GitHub's API, you can extract only certain files or folders like so:

tar -C DESTDIR -xf foo.tar.gz folder1/ folder2/

Do note that the GitHub REST API doesn't provide the ability to download only a subset of data. You have to download a tarball consisting of an entire tree or commit.

bk2204
  • 64,793
  • 6
  • 84
  • 100