1

Context: It is mentioned that Git Credential Managers or Personal Access Tokens is recommended to securely connect to Azure DevOps (SSH would shortly be disabled by security.). Our repository is very large and has a lot of history. Cloning a repository takes time, hence, using "Sync sources" in VSO Build Pipelines delays the pipeline by ~10 minutes. We wanted ability to download parts of source-code (vso git) during VSO build pipelines execution.

Ask: git archive could be used to download specific sub-folders of git repo directly from remote, without a full git clone. How can we download zip/tar from DevOps Git for specific subFolder without git clone & SSH protocol? How could we use the recommended PAT token/ AzureCredManager with git archive remote?

Refs:

git archive --remote="https://<repo>" <branch> <subFolder>
fatal: operation not supported by protocol
  • https://askubuntu.com/a/1074185/601430 git clone with filter, however, there is no documentation on how to enable the preReq settings [uploadpack.allowfilter, uploadpack.allowanysha1inwant] on DevOps Git server.
  • sparse-checkout requires download of git remote add/indexes, which is still costly.
AAATechGuy
  • 385
  • 2
  • 9

1 Answers1

1

Update:

Workaround: Disable the "Get sources" step and get only the source you want by manually executing the according git commands in a script. To do this either use - checkout: none in YAML pipeline or check don't sync source in get source step.

[enter image description here

More details please kindly refer Shayki Abramczyk's excellent reply in this question: Checkout part of a branch in Azure DevOps Pipelines (GetSources)

For Azure DevOps Git repo, there is no way to specify part of files to be downloaded during Get source step in Azure DevOps Pipeline for now.

According to your description, seems your repo is too big or has too many binaries in it. Consider splitting it into smaller repos, or if it has a lot of binaries, using Git-LFS for binaries.

More details please refer this doc-- Manage and store large files in Git

Another workaround is using private agent.If the build is queued on private agent, is set Clean option as false in Get sources step.

enter image description here

After setting Clean option as false, it will download the changed files (instead of all the files) in Get sources step.

Community
  • 1
  • 1
PatrickLu-MSFT
  • 49,478
  • 5
  • 35
  • 62
  • :-/ not ideal, as our repository is legacy, and splitting is huge work. I believe in place of binaries, we do have large amount of files. However, even checking out index (without any code) also takes lot of time. – AAATechGuy Feb 25 '20 at 23:27
  • @Abin OK, you could use a workaround: Disable the "Get sources" step and get only the source you want by **manually executing the according git commands in a script**. To do this either use `- checkout: none` in YAML pipeline or check don't sync source in get source step. More details please see my update part. – PatrickLu-MSFT Feb 26 '20 at 03:39