Gitlab diff between Merge Request Target branch and Source Branch in pipeline

Question

My use involves finding a diff between Merge Request and Master or Release branch (whichever branch I want to diff against) the MR will have CI_MERGE_REQUEST_TARGET_BRANCH set.

CHANGED_DIRS=$(git diff --name-only ${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}...${CI_COMMIT_REF_NAME} | xargs -L1 dirname | uniq | cut -d'/' -f1 |tail -n+2 | uniq)

But it seems that gitlab doesn't recognize the Merge request target branch diff and gives this output.

fatal: ambiguous argument 'master...source': unknown revision or path not in the working tree.

How do I find the diff (which are directories). Appreciate help !

no - but `git fetch` what ? By default gitlab first init the repo and does the git add remote and checkout the branch (that is merge request branch) . Do you mean I should `git fetch origin master` first before diff ? — vgdub, Jul 05 '21 at 21:26
`get fetch` by itself, without arguments, it will fetch all branches. `By default gitlab first init the repo and does the git add remote and checkout the branch (that is merge request branch)` No, kind of, gitlab-ci ends up on detached head without branch checkout, but it depends on configuration. There's even a default number of commits checked in project configuration, I think default is 50. — KamilCuk, Jul 05 '21 at 21:41
ok still got the same error , I am surprised how it works locally but not in gitlab ci , strange ``` * [new branch] master -> origin/master fatal: ambiguous argument 'origin/master...whatever': unknown revision or path not in the working tree. ``` — vgdub, Jul 05 '21 at 21:49
But yes git fetch did got all branches correctly. so locally I assume it must have the same structure as my local — vgdub, Jul 05 '21 at 21:50
`git fetch && git checkout ${CI_MERGE_REQUEST_TARGET_BRANCH_NAME} && git pull` , this worked. — vgdub, Jul 05 '21 at 22:13

score 4 · Answer 1 · answered Dec 29 '21 at 06:56

I solved this error by changing the command from

git diff --name-only ${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}...${CI_COMMIT_REF_NAME}

to

git diff --name-only remotes/origin/${CI_MERGE_REQUEST_TARGET_BRANCH_NAME}...remotes/origin/${CI_COMMIT_REF_NAME}.

It seems that the gitlab runner doesn't have any local branches. So we have to tell it to compare the difference between remote branches.

score 4 · Answer 2 · answered Mar 30 '23 at 23:41

As someone who already mentioned, this is because GitLab-CI does a shallow clone and not a full clone. However, rather than engineering a way to do a full clone or using remote comparisons, I use an inbuilt variable in GitLab-CI : CI_MERGE_REQUEST_DIFF_BASE_SHA.

To get the files use : git diff --name-only ${CI_MERGE_REQUEST_DIFF_BASE_SHA}

CI_MERGE_REQUEST_DIFF_BASE_SHA - The base SHA of the merge request diff. Ref

score 3 · Accepted Answer · answered Jul 06 '21 at 08:39

This is not really a Git issue, but rather a GitLab-CI issue. However, these particular issues crop up in multiple CI/CD systems, always with the same general reason:

Making a full clone of a large repository is slow.
CI systems often start by cloning a repository.
Hence, CI systems often start by making a less-than-full clone of the repository.
Git offered just two ways to do this in the past:¹ shallow clones and single-branch clones. In fact, they tend to go together: making a shallow clone, with git clone --depth number, also makes a single-branch clone unless you forcibly override Git's default by adding --no-single-branch as well.

Now, the clone command in any given CI/CD system may be controllable, or may be done by the CI/CD system before any software knobs that are available to you-as-a-developer are provided. If you can affect the command, that's usually the way to go, but if not, you may still be able to correct the problem by running additional Git commands (this again depends on the CI/CD system):

git remote set-branches --add allows you to add various branches to an otherwise single-branch clone;
git fetch --depth number, git fetch --deepen number, and git fetch --unshallow are three ways to alter the depth of an existing shallow clone, or remove the depth restrictions entirely.

If your CI/CD system makes a shallow, single-branch clone, and you need a deeper clone with more remote-tracking names, you can get that using git remote and/or git fetch.

As I haven't used GitLab's CI system, I cannot provide a recipe here, but based on comments, it sounds like they use --depth 50 by default. If you have a depth knob, setting it to zero (0) is the usual way to disable the --depth argument, which also disables the single-branch-ness.

Do note that regardless of how the system clones your repository, you will generally have only one branch name, or perhaps even none (detached HEAD) when cloning via tag. A full clone will have a full set of remote-tracking names, while a single-branch clone will have only one remote-tracking name until you use git remote to adjust this and run a subsequent git fetch to add more remote-tracking names.

With GitHub Actions, the v2 checkout makes a shallow single-branch clone by default, while the v1 checkout makes a full clone.

¹There is now a third method, the so-called partial clone with promisor remotes. This is arguably the right way for a CI system to do the trick, but partial clones are not very good yet, for a lot of reasons, so this probably has to wait for improvements to the implementation.

Thanks @torek , I did exactly the depth on fetch so now I do `git fetch --depth 30` for my master/release branches. The problem here was more on Gitlab exactly as in your Merge Requests(i.e. Pull requests) they mark this as a new `refs/merge_requests/:id/` AND if you want to do `git diff merge_request....master` it doesn't recognize the master because it think there is only one branch , so on the container where the CI is running I had to locally fetch and clone the master/release branch to get a proper diff for me to work . But your answer is perfect on clone. — vgdub, Jul 07 '21 at 12:09

Gitlab diff between Merge Request Target branch and Source Branch in pipeline

3 Answers3

Linked