Questions tagged [git-sync]
15 questions
3
votes
2 answers
Git-sync sidecar container is not syncing GitHub repo DAGS into Airflow Kubernetes cluster properly
Im attempting to incorporate git-sync sidecar container into my Airflow deployment yaml so my private Github repo gets synced to my Airflow Kubernetes env every time I make a change in the repo.
So far, it successfully creates a git-sync container…
jorgeavelar98
- 75
- 1
- 9
2
votes
2 answers
Is there a different way to mount DAGs from private repo to Airflow on Kubernetes with GitSync without using SSH keys?
I have an Airflow environment (v2.4.3) on Kubernetes and I want to sync it with a private git repo so that any changes I make to DAGs in my master branch get automatically picked up by my Airflow environment.
According to Airflow documentation, I…
jorgeavelar98
- 75
- 1
- 9
2
votes
0 answers
Where can I see the logs for failed airflow scheduler deployments?
I'm deploying Airflow with Helm (I'm using Rancher to do so)
I've set up git-sync to pull the DAGs from a private repo. Everything is working fine.
Now I want to switch to a different repo. I updated repo and subPath
in the Helm configuration
…
Stichiboi
- 79
- 1
- 5
1
vote
0 answers
Passing Airflow Context to init_container in KubernetesPodOperator
Airflow's KubernetesPodOperator provides an init_containers parameter, with which you can specify kubernetes init_containers. However init_containers expects a list of kubernetes.models.V1Container, and I don't see any way to pass airflow context…
ForeverWintr
- 5,492
- 2
- 36
- 65
1
vote
0 answers
Issue Deploying Git Sync DAG to Airflow on Kubernetes
I have been trying to deploy the Git Sync DAG (v3.4.0) to my instance of Airflow (v2.4.1 with helm chart version 1.7.0) running on a kubernetes cluster (v1.23.7+rke2r2).
I followed the deployment instructions from the Airflow documentation which can…
Max Feinberg
- 810
- 1
- 6
- 21
1
vote
1 answer
How to override git-sync sync period
I am using git-sync as a sidecar in Kubernetes to do git-pull and mount the pulled data to shared volume periodically.
Everything works well except GIT_SYNC_PERIOD. I want git sync every 10min, somehow it always use the default value which is…
Musa
- 15
- 1
- 5
1
vote
1 answer
Git push sync throws remote: error: refusing to delete the current default branch 'refs/heads/master'
I am trying to sync my github repo with AWS codecommit. while trying to do git push sync --mirror, I am not sure why it's trying to delete. Same code works fine for the master branch, when I shift to preprod branch it fails with the below…
Eva
- 515
- 4
- 28
1
vote
1 answer
Docker container with git-sync returns Host key verification failed when doing git fetch
My docker-compose.yml looks like:
gitsync:
image: openweb/git-sync:0.0.1
restart: always
command: >
sh -c "ssh-keygen -q -t rsa -N '' -f ~/.ssh/id_rsa &&
ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts &&
…
mrc
- 2,845
- 8
- 39
- 73
0
votes
1 answer
kubernetes Git-sync Could read username for 'https://github.com'
I'm deploying Apache Airflow using the oficial helm chart and wanted to use Git-Sync to feed the dags folder with it. Reading the docs and inspecting the helm value (print below), I used the user<>password alternative
I created the git-credentials…
Lucas Abreu
- 425
- 3
- 14
0
votes
0 answers
Airflow sync not only dags but also custom operators with Git
My Airflow's Git repository has the following structure:
├── dags/
│ └── example-dag1.py
│ └── example-dag2.py
├── include/
│ ├── operators/
│ │ └── my_operator.py
│ └── sensors/
│ …
tomomomo
- 157
- 2
- 13
0
votes
0 answers
Why sync in vscode is working properly after updated python?
I have updated my conda env (python version and every other packages that I have inside it).
After that, I can not sync my commits properly. It gives me to enter password of my git. Even though my account is logged in.
The interesting thing is that…
Bahador Beigomi
- 1
- 2
0
votes
0 answers
How to design a script github sync to gitlab specific destination branch?
Such as:
source repo on github Branch master -> destination repo on gitlab Branch main
I'm newbie for github action.
How to design a script entrypoint.sh or new pipeline or example for me, please.
Note: sorry, my english not strong.
Workflows:…
0
votes
1 answer
git-sync using github personal access token (PAT) for git submodules
I have a Github repo which I configure in git-sync to sync files with my Kubernetes airflow instance by specifying the repo URL with PAT token like below:
https://@github.com/org/dag_repo.git`
(PAT token is inserted by CI/CD and not committed…
Mithun Manohar
- 516
- 1
- 6
- 18
0
votes
1 answer
How to use git-sync image as a sidecar in kubernetes that git pulls periodically
I am trying to use git-sync image as a side car in kubernetes that runs git-pull periodically and mounts cloned data to shared volume.
Everything is working fine when I configure it for sync one time. I want to run it periodically like every 10…
Musa
- 15
- 1
- 5
0
votes
0 answers
Unable to connect with slack webhook from docker container k8s:git-sync
I have a docker container with git-sync k8s.gcr.io/git-sync/git-sync:v3.4.0 that has the following setup:
environment : [
{
name: "GIT_SYNC_BRANCH",
value: "master"
},
{
name: "GIT_SYNC_DEST",
…
mrc
- 2,845
- 8
- 39
- 73