1

My docker-compose.yml looks like:

gitsync:
    image: openweb/git-sync:0.0.1
    restart: always
    command: >
      sh -c "ssh-keygen -q -t rsa -N '' -f ~/.ssh/id_rsa &&
             ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts &&
             chmod 600 ~/.ssh/id_rsa &&
             chmod 600 ~/.ssh/known_hosts"
    environment:
      GIT_SYNC_REPO: git@github.com:{company}/{repo_name}.git
      GIT_SYNC_DEST: /opt/airflow/dags/my_folder
      GIT_SYNC_BRANCH: master
      GIT_SYNC_SSH: 1
      GIT_SSH_KEY_FILE: ~/.ssh/id_rsa
      GIT_SSH_KNOWN_HOSTS_FILE: ~/.ssh/known_hosts
    volumes:
    - ./database_utils:/database_utils
    - ./maintenance:/maintenance
    - ./utils:/utils
    - ./dags:/opt/airflow/dags
    - ./logs:/opt/airflow/logs

The error that I get is:

2022/02/02 16:29:36 error syncing repo: error running command "git fetch origin master": exit status 128: Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

The repository exists because that docker-compose.yml worked when the content was setup to do it through https:

gitsync:
    image: openweb/git-sync:0.0.1
    restart: always
    environment:
      GIT_SYNC_REPO: https://{user}:{personal-access-token}@github.com/{company}/{repo_name}.git
      GIT_SYNC_DEST: /opt/airflow/dags/my_folder
      GIT_SYNC_BRANCH: master
    volumes:
    - ./database_utils:/database_utils
    - ./maintenance:/maintenance
    - ./utils:/utils
    - ./dags:/opt/airflow/dags
    - ./logs:/opt/airflow/logs

I tried to setup the grants to 600 as I read otherwise the key won't be used. Any idea how to fix it?

EDIT: test with dockerfile

I changed docker-compose.yml to:

  gitsync:
    image: my-gitsync-image
    restart: always
    container_name: my-gitsync
    build:
      context: .
      dockerfile: Dockerfile-gitsync
    environment:
      GIT_SYNC_REPO: git@github.com:{company}/{repo}.git
      GIT_SYNC_DEST: /opt/airflow/dags/my_folder
      GIT_SYNC_BRANCH: master
      GIT_SYNC_SSH: 1
      GIT_SSH_KEY_FILE: ~/.ssh/id_rsa
      GIT_SSH_KNOWN_HOSTS_FILE: ~/.ssh/known_hosts

And created a Dockerfile-gitsync as:

FROM openweb/git-sync:0.0.1

    RUN ssh-keygen -q -t rsa -N '' -f ~/.ssh/id_rsa
    RUN ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
    RUN chmod 600 ~/.ssh/id_rsa
    RUN chmod 600 ~/.ssh/known_hosts

Now the error is:

2022/02/02 17:53:41 error syncing repo: error running command "git fetch origin master": exit status 128: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

So I guess that know known host is working but ssh key doesn't like it. I tried doing:

RUN ssh-keygen -q -t rsa -N '' -f ~/.ssh/id_rsa -C "my_company@email.com"

But doesn't work neither.

mrc
  • 2,845
  • 8
  • 39
  • 73
  • Does `ssh-keyscan` (and the entire `command`) run before `git fetch`? – phd Feb 02 '22 at 17:36
  • Don't think so. I tried a different setup and the error has changed. I update the main post again – mrc Feb 02 '22 at 18:08
  • 1
    After `ssh-keygen` and before `git fetch` you have to upload the newly created public key to the Github account. – phd Feb 02 '22 at 21:04
  • 2
    The `ssh-keygen` command will generate a *new* key-pair (public and private) into the selected files, but once generated, the public key must be given to the server Git host so that the server host will accept the client's public key (after verifying it). What step copies the newly generated public key to github? – torek Feb 03 '22 at 09:58
  • thats right @torek. Thanks – mrc Feb 03 '22 at 18:33
  • did you get this to work? I'd like to swap from NFS to using git-sync – trench Mar 26 '22 at 16:23
  • yes @trench it's working fine now. – mrc Mar 28 '22 at 06:27

1 Answers1

0
$ ssh-keyscan -t rsa https://github.com/
getaddrinfo https://github.com/: Name or service not known

SSH doesn't know anything about https://github.com/. It must be just host, not an URL (not to say that SSH and HTTPS URL contradicts a bit, right?)

$ ssh-keyscan github.com
# github.com:22 SSH-2.0-babeld-e47cd09f
github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==
# github.com:22 SSH-2.0-babeld-e47cd09f
github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=
# github.com:22 SSH-2.0-babeld-e47cd09f
github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl
# github.com:22 SSH-2.0-babeld-e47cd09f
# github.com:22 SSH-2.0-babeld-e47cd09f
phd
  • 82,685
  • 13
  • 120
  • 165
  • you are right, doesnt make sense. Despite chaning it the issue stills there with the same error trace. I update the main post. – mrc Feb 02 '22 at 17:12