TL,DR: Authentication problems with SourceTree, gitlab.com and TeamCity when I push/pull or build
Read further for explanations, context and what I already tried:
Maybe I'm exagarating, but I'm desperate. For some reasons I have some authentication problems between any Version Control GUI (SourceTree, Gitkraken) and my private respositories, which are hosted on gitlab.com for the last week.
I'll try to give as much context as possible, hoping for any kind of new hints, assistance or solutions:
I have a simple account on gitlab.com and host different repositories which I'm always the owner. I work with SourceTree to version control the projects I'm working with. I work on a Windows 10 machine and since I work on games I also utilize gitLFS. For years I never had problems regarding any type of authentication.
My SourceTree setup is setup quite simple: there is one account, which is my gitlab account. The protocol was set to HTTPS and I connected the account with the PersonalAccessToken (PAT) I created on gitlab.com This worked for as long as I can remember.
Now last week things changed. At some point I couldn't push or pull anything anymore and I always got this error:
Forbidden fatal: unable to access [Repo-URL] The requested URL returned error: 403
I absolutely did not understand what was happening. THEN I had an ominous thought. What changed? The ONLY thing that changed in my environment was the internet connection.
Now my normal internet setup looks like this:
Standard Router -> Ethernet cable to Switch 1 -> Ethernet Cables to a bunch of devices like my Work PC
But for work related reasons I needed to extend my setup with a router which is able to connect with one of my vServers through VPN.
So now my setup looks like this:
-> Ethernet cable to VPN Router -> Ethernet cable to Switch 2 -> Ethernet to devices which need to be connected to that VPN
Standard Router -
-> Ethernet cable to Switch 1 -> Ethernet Cables to a bunch of devices who don't need to be connected to the VPN
And for specific parts of the work my PC needs also to be on the VPN Router connection, which is why I just plugged the LAN cable into Switch 2. (This setup works great, although still a bit of a hassle swapping my PC LAN Cable manually, but it works and for now it was fine.)
I used this setup for the last week and my PC was connected to the VPN router while I was working. This is the only thing that changed in my environment and I thought is this change responsible for me not being able to push/pull to or from my remotes? That my workstation is connected to a VPN? But why? THen I swapped the slots again: LAN Cable from Switch 2 (the VPN connection) to Switch 1 (the standard connection) and lo and behold - I could push/pull again with SourceTree. What a relief. But about half an hour later it didn't work anymore. Checked gitlab status - everything seems to work fine. Then I got curious - swapped the cables again: LAN Cable from Switch 1 (standard connection) to Switch 2 (VPN connection) -> I could push/pull again with SourceTree, also only about for half an hour. What?
Now I started to worry. I tried everything I found on the internet when I typed that error from above in any search engines:
- log in and out from the gitlab.com web UI
- delete the credentials in the Windows Credential Manager
- delete the account in SourceTree and add it again
- update SourceTree
- downgrade SourceTree
Nothing helped. I always got this unable to access error.
Then my naive wish for a solution told me: "Throw money at the problem". I went and downloaded the trial version of Gitkraken. I worked with Gitkraken a couple of times in the past and was quite fond of it and thought: if this will work out of the box I'll gladly pay whatever they need me to pay to be able to work with private repositories.
Downloaded, quickly setup, cloned the repositores (which...worked?) and could push/pull from it. Yeah! Let's delete SourceTree and never go back right? No, let's first wait half an hour and see annnnnd it doesn't work anymore. Same access unable problems.
Now I went back to SourceTree (since I'm more familiar with it) and did one last thing: Using SSH instead of HTTPS. I created an public and private key with SourceTree, added it to my gitlab.com account, updated the credentials in SourceTree.
This...kinda works. In order to get this to work I always have to add my key to the Putty Authentication Agent (can I automate this, bit annoying to add the keys manually?)
Now if I pull everything seems fine. If I push though I get two outcomes.
Outcome 1:
git -c diff.mnemonicprefix=false -c core.quotepath=false --no-optional-locks push -v --tags origin develop:develop Pushing to gitlab.com:PROJECTPATH.git Locking support detected on remote "origin". Consider enabling it with: $ git config lfs.https://gitlab.com/PROJECTPATH.git/info/lfs.locksverify true
remote: remote: To create a merge request for develop, visit: remote: https://gitlab.com/PROJECTPATH/-/merge_requests/new?merge_request%5Bsource_branch%5D=develop remote:
To gitlab.com:PROJECTPATH.git 32c7b14..854da2a develop -> develop updating local tracking ref 'refs/remotes/origin/develop'
I feel that I never had this "issue" with the locking support, and I also don't know why it tells me what to do if I want to make a merge request? It feels like the system thinkgs I'm not the owner of this repository but a contributor? But overall this seems harmless, no? It gets pushed, though.
Outcome 2:
git -c diff.mnemonicprefix=false -c core.quotepath=false --no-optional-locks push -v --tags origin develop:develop Pushing to gitlab.com:PROJECTPATH.git WARNING: Authentication error: Authentication required: Authorization error: https://gitlab.com/PROJECTPATH.git/info/lfs/locks/verify Check that you have proper access to the repository
remote: remote: To create a merge request for develop, visit: remote: https://gitlab.com/PROJECTPATH/-/merge_requests/new?merge_request%5Bsource_branch%5D=develop remote:
To gitlab.com:PROJECTPATH.git 854da2a..4194e8a develop -> develop updating local tracking ref 'refs/remotes/origin/develop'
Completed successfully.
Once again an authentication error related. It seems it's related to LFS, that's why I also tried to remove and re-install git lfs, but it did not help. BUT it still pushes through. Which is not that bad.
BUT!
I use TeamCity for my buildpipeline and of course I needed to change some things there to get it to work with SSH, but that seemed to by straight forward from what I can tell. But when I run a configuration it starts and stops with an error like this:
Failed to perform checkout on agent: '"C:\Program Files\Git\bin\git.exe" -c credential.helper= checkout -q -f develop' command failed. exit code: 128 stderr: Downloading Project_WFG/Assets/ArticyImporter/ArticyRuntime.dll (178 KB) Error downloading object: Project_WFG/Assets/ArticyImporter/ArticyRuntime.dll (2b1559d): Smudge error: Error downloading Project_WFG/Assets/ArticyImporter/ArticyRuntime.dll (2b1559d3dc9b656cbe7531365c2c57a214dfec5b0ed9f1565b28a8242a043880): batch response: Authorization error: https://gitlab.com/PROJECTPATH.git/info/lfs/objects/batch Check that you have proper access to the repository
Errors logged to D:\Programme\General\TeamCity\buildAgent\system\git\git-855A7B84.git\lfs\logs\20211010T110253.003946.log Use git lfs logs last to view the log. error: external filter 'git-lfs filter-process' failed
So yeah - I'm desperate and I don't know what to do anymore. I really appreciate any hint or help.
