How to properly setup a two-way sync between Azure DevOps and GitHub

Question

I want to create a two-way sync between Azure DevOps and GitHub by doing the following:

1. Create an Azure DevOps pipeline with a CI trigger that push changes from Azure DevOps repo to a branch in GitHub
2. Create a second pipeline that listens for changes from GitHub and pulls them back to a branch in Azure DevOps

Here are the commands that I'm using for both pipeline jobs:

git clone --mirror <source>
cd <repo name>
git remote add --mirror=fetch target <target>
mkdir ../workdir
git fetch origin
git --work-tree=../workdir/ checkout <branch name>
git push target <branch name>

When I test the pipeline I end up with an error like this:

! [rejected]        <branch name> -> <branch name> (fetch first)
error: failed to push some refs to 'git@github.com:<target>/<target repo>.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

I am by no means a git expert, so any help is very much appreciated.

UPDATE # 1

The captures below show the pipeline that I've built to push changes to GitHub (based on this answer), but it still isn't working.

This is the new error:

Cloning into '<repo>'...
Switched to a new branch '<branch name>'
Branch '<branch name>' set up to track remote branch '<branch name>' from 'origin'.
warning: adding embedded git repository: <repo>
hint: You've added another git repository inside your current repository.
hint: Clones of the outer repository will not contain the contents of
hint: the embedded repository and will not know how to obtain it.
hint: If you meant to add a submodule, use:
hint: 
hint:   git submodule add <url> <repo>
hint: 
hint: If you added this path by mistake, you can remove it from the
hint: index with:
hint: 
hint:   git rm --cached <repo>
hint: 
hint: See "git help submodule" for more information.
[<branch name> XXXXXXX] Update from Azure DevOps
 1 file changed, 1 insertion(+)
 create mode 160000 <repo>
To git@github.com:<org>/<repo>.git
 ! [rejected]        <branch name> -> <branch name> (fetch first)
error: failed to push some refs to 'git@github.com:<org>/<repo>.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

UPDATE # 2

I updated the script (shown below) to pull changes from GitHub to Azure DevOps based on the updated answer, but it still isn't working.

git clone git@github.com:XXX/XXX.git 

cd <repo folder>

git config --global user.email "XXX"
git config --global user.name "XXX"

git checkout <source branch>
git add .
git commit -m "Pull from GitHub"
git push https://{AzureDevopsPAT}@dev.azure.com/{org}/{pro}/_git/XXX.git <target branch>

This is the error:

Cloning into '<repo>'...
Warning: Permanently added the RSA host key for IP address '140.82.113.4' to the list of known hosts.
Already on '<source branch>'
Your branch is up to date with 'origin/<source branch>'.
On branch <source branch>
Your branch is up to date with 'origin/<source branch>'.

nothing to commit, working tree clean
error: src refspec <target branch> does not match any
error: failed to push some refs to '***'

Answer 1

if I also make a change in GitHub, can I get that pushed back to Azure DevOps?

For this issue , first you can create new build definition:

In command line tasks, first step setup your info using 2 command:

git config --global user.email ""
git config --global user.name ""

Second step we use this command to loop through all branch in github:

git branch -r | grep -v '\->' | while read remote; do git branch --track "${remote#origin/}" "$remote"; done

Last step to push to all branch:

git push https://$(VSTSToken)@dev.azure.com/xxx/_git/xxx -u --all

In order to kick the build every time there is a code change we need to go to trigger tab and setup like this:

Regarding syncing azure devops repo to github repo, please refer to my previous answer.

Update:

For a single branch,push update from github repo to azure devops repo,see below command:

git clone https://github.com/XXX/XXX.git
git config --global user.name "XXX"
git checkout master
git add .
git commit -m "abc"
git push https://{AzureDevopsPAT}@dev.azure.com/{org}/{pro}/_git/delete.git

Answer 2

I've done similar scenario using Webhooks with Jenkins.

Below sync private repo on GitHub with private repo on Azure Repos

On GitHub:

1- Add SSH Key pair authentication to GitHub using ssh-keygen, copy the public key to Settings -> SSH and GPG keys -> New SSH key.

2- From Github repo -> Settings -> Webhooks -> Add Webhook -> in Payload URL provide http://<your-jenkins-public-ip>:<port>/github-webhook/, select rest of parameters as required and save. I set Content type: application/json, provided secret predefined in repo secrets. in Which events would you like to trigger this webhook? I chose "Send me everything". Make sure it's verified after save.

On Azure DevOps:

3- Create Personal Access Token (PAT) on Azure DevOps profile, copy it and keep it safe.

On Jenkins:

4- Configure Jenkins freestyle project, in "Source Code Management" configure Git with "SSH Username with private key", provide your GitHub username and the privet key you generated in step1.

5- In "Additional Behaviors" tab choose "Wipe out repository & force clone". This enforce wipe and clone every time to avoid errors.

6- In "Build Triggers" tab choose "GitHub hook trigger for GITScm polling".

7- In "Build" tab add below scrip in "Execute shell"

# Now Jenkins already accessed the repo (source) and cloned it using credentials provided in the "Source Code Management" section above.
# Now checkout the branch you want to trigger from
git checkout main

# Our build is to push these updates to a remote repo, in this case "Azure Repos".
# Now, point to the remote repo,
git config remote.origin.url https://<username>@dev.azure.com/<username>/<org_name>/_git/<proj_name>

# Now checkout the branch you want to push to
git checkout main

# Push with personal access token (PAT) configured on Azure DevOps platform.
git push --force https://<Azure_PAT>@dev.azure.com/<username>/<org_name>/_git/<proj_name>

8- Push changes to your GitHub repo & watch Jenkins Build History and its impact on Azure Repos.

You might follow reverse steps on Azure Pipleline for the reverse process.