0

When I run terraform plan, I get errors like:

╷
│ Error: Failed to download module
│
│ Could not download module "foo" (main.tf:123) source code from
│ "git@github.com:my-company/some-repo": error downloading
│ 'ssh://git@github.com/my-company/some-repo': /usr/bin/git exited with 128:
│ Cloning into '.terraform/modules/foo'
│ Host key verification failed.
│ fatal: Could not read from remote repository.
│
│ Please make sure you have the correct access rights
│ and the repository exists.

The Github SSH key is listed in ssh-add -l and I am able to clone ssh://git@github.com/my-company/some-repo. The repo is part of my company's Github org, and it requires SSO on Github to access.

I found https://support.hashicorp.com/hc/en-us/articles/360041922633--Host-key-verification-failed-error-in-a-Terraform-Enterprise-run-when-attempting-to-ingress-Terraform-modules-via-Git-over-SSH- but it didn't really help. The page is vague but from what I can tell I've already done everything they suggest.

Does anyone know a solution to this?

Dommondke
  • 307
  • 1
  • 8
  • Do you get any additional info if you set `TF_LOG=debug` – Chris Doyle Oct 27 '22 at 20:31
  • @phd Terraform is a tool for managing cloud infrastructure, which appears to call git internally for its own dependencies, with different settings than my system git settings. Therefore, my own git configuration is already working and I have no need for the solutions in those links. Terraform's git settings are the issue. – Dommondke Oct 27 '22 at 23:22
  • 2
    Doesn't matter. Terraform uses Git, Git uses SSH, so you have to understand the difference between SSH Host keys and SSH User keys, and you need to put SSH Host key of the remote host into `~/.ssh/known_hosts` for Terraform. SSH Host key in `~/.ssh/known_hosts` is the only thing that matters. – phd Oct 27 '22 at 23:27
  • @phd The host key is already in `~/.ssh/known_hosts`. – Dommondke Oct 27 '22 at 23:37
  • 1
    Does your SSO solution involve overriding the `GIT_SSH_COMMAND` environment variable to be something other than `ssh`? As far as I know, overriding that environment variable is the only unusual thing Terraform does when it runs `git clone ...`. – Martin Atkins Oct 28 '22 at 01:02
  • 3
    (The same would apply to overriding the SSH command in your Git configuration, since the environment variable takes precedence over the Git configuration on disk) – Martin Atkins Oct 28 '22 at 01:04
  • 1
    "Host key verification failed" indicates that the host key that ssh *got* doesn't match the host key *in the known-hosts file*. So even if there is a host key there, it's apparently the wrong one. You'll need to debug that part to make progress. – torek Oct 28 '22 at 04:11
  • @MartinAtkins My SSO requires that, every few days, I navigate to Github in a browser and click a link that says "login with single sign-on". After that, all git commands in my terminal work normally. If the host key was missing from `~/.ssh/known_hosts` then I would get an error when I run `git clone`, but I don't. Perhaps Terraform does not use my regular `~/.ssh/known_hosts` but uses some other files - in which case, how do I find out what it's using? – Dommondke Oct 28 '22 at 15:50
  • I don't know of anything Terraform does to explicitly override the known hosts directory, but it _does_ set the environment variable `GIT_SSH_COMMAND=ssh` before running `git`, and so if you normally have Git configured to run `ssh` in a different way (e.g. with some extra arguments that might change how it treats the "known hosts" file) then those would not be active when Terraform is the one running `git`. – Martin Atkins Oct 28 '22 at 23:00
  • I'm not familiar with this GitHub SSH SSO mechanism so I'm mainly just guessing here based on what I know about how Terraform runs Git, but if you can link to some documentation from GitHub on exactly how this SSO mechanism is set up in the SSH client configuration then I'd love to review it and see it includes anything that might conflict with the way Terraform runs Git (specifically: the way Terraform tells Git to run SSH) – Martin Atkins Oct 28 '22 at 23:02

1 Answers1

2

If you are able to clone ssh://git@github.com/my-company/some-repo without error message, but Terraform is not, that means the terraform plan is executed with a different account (or different $HOME as in here) than your regular account.

If it was done with the same account, it would find the Host key in $HOME/.ssh/known_hosts, as Git does through SSH when you are cloning your repository with the GitHub SSH URL.

Depending on its execution environment (like, for instance, a Jenkins pipeline), you would need to add the Host key.

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • A different `$HOME` would indeed explain it. I am running Terraform from my laptop, not Jenkins. How do I figure out which `$HOME` Terraform is using? – Dommondke Oct 28 '22 at 15:31
  • @Dommondke You can check its [debug log](https://www.suse.com/support/kb/doc/?id=000020022) and see if there is any clue there. – VonC Oct 28 '22 at 16:58