1

I use following steps in Jenkinsfile to push tags to github

sshagent (credentials: ['jenkins']) {
sh """
    git config --add url."git@github.com:".insteadOf "https://github.com/"
    git config user.name 'Jenkins'
    git config user.email '$$$+###@users.noreply.github.com'
    git tag -a dimToOff_v"${PRETTY_BUILD_VERSION}"_"${BUILD_NUMBER}" ${COMMIT_ID} -m "${JENKINS_TAG_MESSAGE}"
    git push --tags
"""
}

This stopped working recently with following error for git push --tags

Host key verification failed.
fatal: Could not read from remote repository.

I followed the steps mentioned here, but it didn't solve the issue.

Also tried to set GIT_SSH_COMMAND=ssh -Tvv, then I got following output.

 + export GIT_SSH_COMMAND=ssh -Tvv
 + true
 + true
 + git config --add url.git@github.com:.insteadOf https://github.com/
 + git config user.name Jenkins
 + git config user.email ***+###@users.noreply.github.com
 + git tag -a dimToOff_v0.0.0_18 0dcd640861c996bb23fffe7d09353085a150d5fb -m By Jenkins
 + cat /home/jenkins/.ssh/known_hosts
 |1|ALkYORymbpagfGBK9E5gJ8woD4Q=|TpduH1siXgLBGdrr+LIXqRymiT4= ssh-rsa ***
 |1|VfTJRk4WA1frUmmU6aHuphGFhWg=|sl7TtxUV8s9Dw8KC6zYxjYgESbc= ssh-rsa ***
 |1|Vbfb9vRoSIzMl+1LfrD7DVOTD9w=|AGxlbOuxn+83KCtiMvaKKslq8XY= ssh-rsa ***
 |1|ND2ZisnFqJ1Z6lK6WzutcO5ZNqw=|sq4nIsV6pBWZ4P27EJHgq1rVkUs= ssh-rsa ***
 |1|bIQPH+rXgCL9Y79j7mXWrVeurxU=|dapFBijhdY4Q1GAO38e28popmFw= ssh-rsa ***
 |1|2qiJH9WIN45TcVwVV3abSeaOu+Y=|wCc07ew86oGV8JZlZCfD/nxUC10= ssh-rsa ***
 |1|1YIO0QnRQqmAEUnZpkU7C7fKoXw=|aKQev539WQiIxOPCOUHLUBWMFEE= ssh-rsa ***
 |1|A4okl1ZD/XEPYK+28Y7QvFKpwt8=|Q7Gvj/MtVMeXKM9R9E9lTTgQ0RY= ssh-rsa ***
 |1|dEKvmuFW1eRhNd3xK90iJhyGw/Q=|/+bFyB5TPDNtynliKh4h2rHKVVI= ssh-rsa ***
 |1|8rZZylU3IElRZ4xZeyxYB6QFMb8=|x10+YhzXeMIXH33yLy8VV/1K4IA= ssh-rsa ***
 |1|QOBmqHVPJXsjrShtEbrEvPHCz28=|K9/Esrj6Ku1FQ1PesQBIJJY9lcA= ecdsa-sha2-nistp256 ***
 + ssh-keygen -R github.com
 # Host github.com found: line 11
 /home/jenkins/.ssh/known_hosts updated.
 Original contents retained as /home/jenkins/.ssh/known_hosts.old
 + ssh-keyscan github.com
 # github.com:22 SSH-2.0-babeld-f06bbde2
 # github.com:22 SSH-2.0-babeld-f06bbde2
 # github.com:22 SSH-2.0-babeld-f06bbde2
+ git push --tags
OpenSSH_7.6p1 Ubuntu-4ubuntu0.5, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /home/jenkins/.ssh/config
debug1: /home/jenkins/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "github.com" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to github.com [192.30.255.112] port 22.
debug1: Connection established.
debug1: identity file /home/jenkins/.ssh/id_rsa type 0
debug1: key_load_public: No such file or directory
debug1: identity file /home/jenkins/.ssh/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.5
debug1: Remote protocol version 2.0, remote software version babeld-f06bbde2
debug1: no match: babeld-f06bbde2
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to github.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256
debug2: host key algorithms: ssh-ed25519,ecdsa-sha2-nistp256,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
debug2: MACs ctos: hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256
debug2: MACs stoc: hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-256
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:p2QAMXNIC1TJYWeIOttrVc98/R1BUFWu3/LiyKgUfQM
debug1: Host 'github.com' is known and matches the ECDSA host key.
debug1: Found key in /home/jenkins/.ssh/known_hosts:12
debug1: read_passphrase: can't open /dev/tty: No such device or address
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Any help on how to fix this ?

roy
  • 6,344
  • 24
  • 92
  • 174
  • https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/ – larsks Apr 06 '23 at 19:26
  • Did you follow those steps as the Jenkins user? – Jim Redmond Apr 06 '23 at 20:05
  • Yes, I performed it as Jenkins user. – roy Apr 06 '23 at 20:05
  • It looks like you're failing host key verification. This is usually because you have a bad/old entry in your known_hosts file. You can either fix the entry or [disable host key verification altogether](https://stackoverflow.com/a/28527476/5747944) (which obviously has some security implications). – sytech Apr 06 '23 at 20:39
  • What's the rest of the message, before `Host key verification failed`? – Jim Redmond Apr 06 '23 at 21:09
  • `Host key verification failed` is the first error message. All previous commands are succeeding. – roy Apr 10 '23 at 17:25

2 Answers2

1

Try and inspect your ~/.ssh/known_hosts during your job, before the git push step.

sh "cat ~/.ssh/known_hosts"

If you see any GitHub-related line, you need to remove them with

sh "ssh-keyscan -R github.com"

before adding

sh "ssh-keyscan github.com >> ~/.ssh/known_hosts"

Try also, in your git sh step, to add

export GIT_SSH_COMMAND='ssh -Tvv'

That way, you will see exactly what file is considered when Git is trying to connect through SSH to github.com.

debug1: read_passphrase: can't open /dev/tty: No such device or address
Host key verification failed.

I would recommend using a non-encrypted private key (you can remove the passphrase of your existing key)

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • added output for this suggestion to main question. – roy Apr 11 '23 at 17:45
  • @roy At this point, you are good to go. You just need to use a private key *without* passphrase (not encrypted). – VonC Apr 11 '23 at 18:05
  • My key for github is non-encrypted. Still face this issue. – roy Apr 18 '23 at 19:47
  • 1
    @roy That would mean the issue is in the `~/.ssh/known_hosts` file, where the remote server (github.com)'s public key is stored along with a fingerprint and an optional passphrase. Try and empty that file, adding only `sh "ssh-keyscan github.com > ~/.ssh/known_hosts"` (one `>` instead of `>>`, to override everything) – VonC Apr 18 '23 at 21:06
0

Git commands will fail if the github server isn't in your build node's known hosts list.

Try adding this step before running git commands:

   sh "ssh-keyscan github.com >> ~/.ssh/known_hosts"
Vasiliki Siakka
  • 1,185
  • 8
  • 15