Airflow SFTPHook - No hostkey for host found

Question

I'm trying to use the Airflow SFTPHook by passing in a ssh_conn_id and I'm getting an error:

No hostkey for host myhostname found.

Using the SFTPOperator with the same ssh_conn_id however is working fine. How can I resolve this error?

SFTPHook should take `ftp_conn_id` instead of `ssh_conn_id`. — Ryan Yuan, Jan 29 '19 at 00:49

bastien-r · Answer 1 · 2020-09-03T08:53:51.533

3

Just had this issue, the simple trick is to keep your SSH connector inside airflow and to add the following in the "Extra" field :

{"no_host_key_check": true}

Hope it helps !

Edit : Indeed, it allows the man-in-the-middle attack, so even if it helps temporarily, you should get the ssh fingerprint and allow it

edited Sep 03 '20 at 08:53

answered Feb 11 '19 at 13:29

bastien-r

2

Disabling the host key check is not recommended as it leaves you vulnerable to man-in-the-middle attacks. – Cookie Monster Feb 11 '19 at 14:35
@CookieMonster As I understand it is indeed not recommended to do this in production. However, is there an alternative method described anywhere? Running a managed Airflow cluster using Cloud Composer, I do not see where I can modify the `~/.ssh/known_hosts file`. – Pascal Delange Apr 27 '21 at 13:30

score 0 · Answer 2 · answered Jan 26 '19 at 22:20

0

The SFTPOperators uses SSHHook. Hence, you should use SSHHook instead.

answered Jan 26 '19 at 22:20

kaxil

1

I experienced the same issue with the SSHHook. It seems that Paramiko was looking for the known_hosts file in this path: /home/airflow/.ssh/known_hosts. In the end I wasn't able to figure out how to get it working since I have no access to the path from Composer and I resorted to writing my own implementation using pysftp. – Cookie Monster Feb 11 '19 at 14:26

2 Answers2