0

I have a script that automates some large pull-request tasks that we do. One thing I do is to try to automatically sync two of the branches we use. I used to use a token to do a git fetch but had to change it recently due to some policy changes.

So what I'm doing now is the below.

subprocess.check_call(['git', 'fetch', f'https://{username}:{password}@{repo_url}'], cwd=repo_path)

But, I've noticed that this doesn't always show when my local branch is out of sync with the remote branch.

Question

My question is this, what is different about git fetch versus when I use my username, password, and repo_url that causes it not to notice remote changes when I use my username and credentials to do the fetch? What should I be doing different? And what should I be using instead to guarantee that my script notices when the remote branch is out of sync?

Further Details

I've tried scripting the git fetch all by itself, and it works just fine, only that it requires me putting in my username and password again.

subprocess.check_call(['git', 'fetch'], cwd=repo_path)
HereForTheAnswers
  • 65
  • 5
  • Could you try a credential manager? – Cole Tierney Mar 29 '21 at 13:24
  • I'd prefer not too. Our IT really frowns upon storing user credentials. – HereForTheAnswers Mar 29 '21 at 13:29
  • Can you use ssh with key authentcation? – Cole Tierney Mar 29 '21 at 14:06
  • What, precisely, does "doesn't always work" mean? Does the `git fetch` call fail? Does it succeed and update *some* remote-tracking names but not *all* remote-tracking names? Does it do nothing at all? What, if any, output does it produce? Since `check_call` makes sure that the exit status is zero (otherwise raises an exception), does it finish without exceptions, or does it end with an exception? – torek Mar 29 '21 at 23:44
  • (Ideally, you should include a reproducer, but if it's a private repo, that's obviously pretty difficult!) – torek Mar 29 '21 at 23:45
  • @torek, It doesn't update all remote tracking names when I make change and push it from another system. – HereForTheAnswers Mar 30 '21 at 12:53
  • @torek, And yeah, sorry I can't provide the repo because it's something for work, so the repo is private. I did post an answer as to what worked for me. Feel free to add on, or explain why what I did works. Some of it baffles me honestly as to why I had to add an explicit refspec when doing it scripted, but a manual git fetch worked just fine. – HereForTheAnswers Mar 30 '21 at 13:24
  • If you're going through a large hosting site (GitHub, Bitbucket, etc) I wonder if it might just be a timing issue where some front end is caching when it shouldn't. But even then an explicit refspec should not change the script's behavior. Old (pre-1.8.4) Git versions behave badly with partial refspecs, but that also does not depend on scripted-vs-manual. – torek Mar 30 '21 at 18:38

1 Answers1

0

For anybody else who runs into this issue:

You want to do something like this, where master is the branch you want to update

subprocess.check_call(['git', 'fetch', f'https://{username}:{password}@{repo_url}','origin', master:refs/remotes/origin/master'], cwd=repo_path)

The issue seems to be related to FETCH_HEAD not always updating when a git fetch is run. Therefore changes that happen remotely don't always get pulled in when a git fetch is run. Why that would work when run manually but not when scripted is a mystery to me.

See also this question here and this one which led me to a solution.

From the first answer I found this:

// Pulls the remote 'master' branch down to the local 'master' branch
git pull origin master:refs/remotes/origin/master

From the second answer I was able to pull this line:

As a solution, if git fetch remote_name branch_name is used only this specific branch is fetched and only that single line will appear in the contents of .git/FETCH_HEAD, making the FETCH_HEAD reference always correct.

#Will only fetch branchone
git fetch origin branchone

Putting those two together I came up with the syntax that works. I am guaranteed to get the updates that show when a local branch is out of sync by specifying the refspec explicitely which is this part master:refs/remotes/origin/master. Replace master with your branch name and it will always update your local branch to show when it is out of sync.

The third argument f'https://{username}:{password}@{repo_url}' is only needed if you want to collect the username and password at the beginning of your script and use them in any git commands that need it. Otherwise it can be omitted.

HereForTheAnswers
  • 65
  • 5