1

I'm writing a script to regularly transfer files between two SFTP servers, and I am using Paramiko for this. The server is authenticated using private key + password. However, I have been struggling for many days to make this work and have been unsuccessful in just getting to authenticate to the SFTP server.

The code snippet that I have to make the connection is this.

 1   import paramiko
   1 import logging
   2
   3 if __name__ == "__main__":
   4     print(logging.DEBUG)
   5     logging.basicConfig()
   6     logging.getLogger("paramiko").setLevel(logging.DEBUG)
   7     logging.getLogger("paramiko.transport").setLevel(logging.DEBUG)
   8     client = paramiko.SSHClient()
   9     client.load_system_host_keys()
  10     client.set_missing_host_key_policy(paramiko.AutoAddPolicy)
  11
  12     client.connect(
  13         hostname='host_ip',
  14         port=123,
  15         username='some_user',
  16         password='some_pass',
  17         key_filename='path_to_priv_key',
  18         allow_agent=False,
  19         look_for_keys=False)
  20
  21     client.open_sftp()

I am getting the following error.

DEBUG:paramiko.transport:starting thread (client mode): 0xcf48b610
DEBUG:paramiko.transport:Local version/idstring: SSH-2.0-paramiko_2.7.2
DEBUG:paramiko.transport:Remote version/idstring: SSH-2.0-WS_FTP-SSH_8.6.1.1504
INFO:paramiko.transport:Connected (version 2.0, client WS_FTP-SSH_8.6.1.1504)
DEBUG:paramiko.transport:kex algos:['diffie-hellman-group1-sha1', 'diffie-hellman-group14-sha1', 'diffie-hellman-group14-sha256', 'diffie-hellman-group-exchange-sha256'] server key:['ssh-dss', 'ssh-rsa'] client encrypt:['aes256-ctr', 'aes192-ctr', 'aes128-ctr', '3des-cbc', 'blowfish-cbc', 'aes256-cbc', 'aes128-cbc', 'cast128-cbc'] server encrypt:['aes256-ctr', 'aes192-ctr', 'aes128-ctr', '3des-cbc', 'blowfish-cbc', 'aes256-cbc', 'aes128-cbc', 'cast128-cbc'] client mac:['hmac-sha1', 'hmac-sha1-96', 'hmac-md5', 'hmac-md5-96', 'hmac-sha2-256', 'hmac-sha2-384', 'hmac-sha2-512'] server mac:['hmac-sha1', 'hmac-sha1-96', 'hmac-md5', 'hmac-md5-96', 'hmac-sha2-256', 'hmac-sha2-384', 'hmac-sha2-512'] client compress:['none'] server compress:['none'] client lang:[''] server lang:[''] kex follows?False
DEBUG:paramiko.transport:Kex agreed: diffie-hellman-group-exchange-sha256
DEBUG:paramiko.transport:HostKey agreed: ssh-rsa
DEBUG:paramiko.transport:Cipher agreed: aes128-ctr
DEBUG:paramiko.transport:MAC agreed: hmac-sha2-256
DEBUG:paramiko.transport:Compression agreed: none
DEBUG:paramiko.transport:Got server p (2048 bits)
DEBUG:paramiko.transport:kex engine KexGexSHA256 specified hash_algo <built-in function openssl_sha256>
DEBUG:paramiko.transport:Switch to new keys ...
DEBUG:paramiko.transport:Trying discovered key b'db7022e07xxxxxx366618922bacb6e21' in <priv_key_path>
DEBUG:paramiko.transport:userauth is OK
INFO:paramiko.transport:Authentication continues...
DEBUG:paramiko.transport:Methods: ['password']
DEBUG:paramiko.transport:EOF in transport thread
Traceback (most recent call last):
  File "connect.py", line 13, in <module>
    client.connect(
  File "...code/sandbox/sftp/env/lib/python3.8/site-packages/paramiko/client.py", line 435, in connect
    self._auth(
  File "...code/sandbox/sftp/env/lib/python3.8/site-packages/paramiko/client.py", line 764, in _auth
    raise saved_exception
  File "...code/sandbox/sftp/env/lib/python3.8/site-packages/paramiko/client.py", line 751, in _auth
    self._transport.auth_password(username, password)
  File "...code/sandbox/sftp/env/lib/python3.8/site-packages/paramiko/transport.py", line 1509, in auth_password
    return self.auth_handler.wait_for_response(my_event)
  File "...code/sandbox/sftp/env/lib/python3.8/site-packages/paramiko/auth_handler.py", line 236, in wait_for_response
    raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.

I even tried to create the transport layer manually, but to no avail. auth_publickey() successfully does a partial authentication, but anything that I do after that causes the transport to error out with an EOF.

  s = socket.socket()
  s.connect(('host_ip', 123))
  t = paramiko.Transport(s)
  t.connect()

  try:
    print('key auth')
    pkey = paramiko.RSAKey.from_private_key_file(
        'path_to_priv_key')
    print(t.auth_publickey('some_user', pkey))
    print('checking')
    print(t.is_authenticated())

    print('pass auth')
    print(t.auth_password('some_user', 'some_pass'))
    print('checking')
    print(t.is_authenticated())
  except paramiko.AuthenticationException as e:
    print(e)
  except paramiko.BadAuthenticationType as e:
    print(e)

And the logs for this is as below

DEBUG:paramiko.transport:starting thread (client mode): 0x6704b790
DEBUG:paramiko.transport:Local version/idstring: SSH-2.0-paramiko_2.7.2
DEBUG:paramiko.transport:Remote version/idstring: SSH-2.0-WS_FTP-SSH_8.6.1.1504
INFO:paramiko.transport:Connected (version 2.0, client WS_FTP-SSH_8.6.1.1504)
DEBUG:paramiko.transport:kex algos:['diffie-hellman-group1-sha1', 'diffie-hellman-group14-sha1', 'diffie-hellman-group14-sha256', 'diffie-hellman-group-exchange-sha256'] server key:['ssh-dss', 'ssh-rsa'] client encrypt:['aes256-ctr', 'aes192-ctr', 'aes128-ctr', '3des-cbc', 'blowfish-cbc', 'aes256-cbc', 'aes128-cbc', 'cast128-cbc'] server encrypt:['aes256-ctr', 'aes192-ctr', 'aes128-ctr', '3des-cbc', 'blowfish-cbc', 'aes256-cbc', 'aes128-cbc', 'cast128-cbc'] client mac:['hmac-sha1', 'hmac-sha1-96', 'hmac-md5', 'hmac-md5-96', 'hmac-sha2-256', 'hmac-sha2-384', 'hmac-sha2-512'] server mac:['hmac-sha1', 'hmac-sha1-96', 'hmac-md5', 'hmac-md5-96', 'hmac-sha2-256', 'hmac-sha2-384', 'hmac-sha2-512'] client compress:['none'] server compress:['none'] client lang:[''] server lang:[''] kex follows?False
DEBUG:paramiko.transport:Kex agreed: diffie-hellman-group-exchange-sha256
DEBUG:paramiko.transport:HostKey agreed: ssh-rsa
DEBUG:paramiko.transport:Cipher agreed: aes128-ctr
DEBUG:paramiko.transport:MAC agreed: hmac-sha2-256
DEBUG:paramiko.transport:Compression agreed: none
DEBUG:paramiko.transport:Got server p (2048 bits)
DEBUG:paramiko.transport:kex engine KexGexSHA256 specified hash_algo <built-in function openssl_sha256>
DEBUG:paramiko.transport:Switch to new keys ...
key auth
DEBUG:paramiko.transport:userauth is OK
INFO:paramiko.transport:Authentication continues...
DEBUG:paramiko.transport:Methods: ['password']
['password']
checking
False
pass auth
DEBUG:paramiko.transport:EOF in transport thread
Authentication failed.

Can someone help with this please or give any pointers? Is there a lower level approach to doing this if this capability is not supported by Paramiko?

My versions are
Python: 3.8.7
Paramiko: 2.7.2

Update: This works fine when I use sftp on the command line. The verbose logs when I do this are below.

OpenSSH_8.2p1, OpenSSL 1.1.1g  21 Apr 2020
debug1: Reading configuration data ~/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 93: Applying options for *.*
debug1: hostname canonicalisation enabled, will re-parse configuration
debug1: re-parsing configuration
debug1: Reading configuration data ~/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 93: Applying options for *.*
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to host_ip [host_ip] port 123.
debug1: using TCP window size of 4194304 / 4194304
debug1: Connection established.
debug1: identity file ~/.ssh/keyfile type 1
debug1: identity file ~/.ssh/keyfile-cert type -1
debug1: identity file ~/.ssh/identity type -1
debug1: identity file ~/.ssh/identity-cert type -1
debug1: identity file ~/.ssh/localhost/identity type -1
debug1: identity file ~/.ssh/localhost/identity-cert type -1
debug1: identity file ~/.ssh/clusterhost/identity type -1
debug1: identity file ~/.ssh/clusterhost/identity-cert type -1
debug1: identity file ~/.ssh/id_dsa type -1
debug1: identity file ~/.ssh/id_dsa-cert type -1
debug1: identity file ~/.ssh/localhost/id_dsa type -1
debug1: identity file ~/.ssh/localhost/id_dsa-cert type -1
debug1: identity file ~/.ssh/clusterhost/id_dsa type -1
debug1: identity file ~/.ssh/clusterhost/id_dsa-cert type -1
debug1: identity file ~/.ssh/id_rsa type 0
debug1: identity file ~/.ssh/id_rsa-cert type -1
debug1: identity file ~/.ssh/localhost/id_rsa type -1
debug1: identity file ~/.ssh/localhost/id_rsa-cert type -1
debug1: identity file ~/.ssh/clusterhost/id_rsa type -1
debug1: identity file ~/.ssh/clusterhost/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.2
debug1: Remote protocol version 2.0, remote software version WS_FTP-SSH_8.6.1.1504
debug1: no match: WS_FTP-SSH_8.6.1.1504
debug1: Authenticating to host_ip:123 as 'some_user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: diffie-hellman-group-exchange-sha256
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(2048<8192<8192) sent
debug1: got SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: got SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: ssh-rsa SHA256:etDpHOm1sxxxxxxGk42y8peYNjUfHIb3+Y3vu12gkjA
debug1: KRL version 0 generated at 20190904T045457
debug1: Host '[host_ip]:123' is known and matches the RSA host key.
debug1: Found key in ~/.ssh/known_hosts:21
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: ~/.ssh/keyfile DSA SHA256:6FzoJ/xxxxxx+WrlZ4mhZBi0+Q7
Ancc59Wqn+D5k7HE explicit
debug1: Will attempt key: ~/.ssh/identity
debug1: Will attempt key: ~/.ssh/localhost/identity
debug1: Will attempt key: ~/.ssh/clusterhost/identity
debug1: Will attempt key: ~/.ssh/id_dsa
debug1: Will attempt key: ~/.ssh/localhost/id_dsa
debug1: Will attempt key: ~/.ssh/clusterhost/id_dsa
debug1: Will attempt key: ~/.ssh/id_rsa RSA SHA256:k2p9JH9AfcZlxxxxxxsJgBfQ0Fhuvxuukr
KnxxxxviU
debug1: Will attempt key: ~/.ssh/localhost/id_rsa
debug1: Will attempt key: ~/.ssh/clusterhost/id_rsa
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering public key: ~/.ssh/keyfile DSA SHA256:6FzoJ/xxxxxx+WrlZ4mhZBi0
+xxxxxx59Wqn+D5k7HE explicit
debug1: Server accepts key: ~/.ssh/keyfile DSA SHA256:6FzoJ/xxxxxx+WrlZ4mhZBi0+
xxxxxx59Wqn+D5k7HE explicit
Authenticated with partial success.
debug1: Authentications that can continue: password
debug1: Next authentication method: password
debug1: Authentication succeeded (password).
Authenticated to host_ip ([host_ip]:123).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: pledge: network
debug1: Sending subsystem: sftp
Connected to host_ip.
Cannot download non-regular file: /OutPutFolder/202009/2020-09-29/
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 2 clearing O_NONBLOCK
Transferred: sent 49640, received 25837144 bytes, in 60.3 seconds
Bytes per second: sent 823.2, received 428468.5
debug1: Exit status -1

The part that is working here which is failing on Paramiko is

debug1: Authentications that can continue: password
debug1: Next authentication method: password
debug1: Authentication succeeded (password).

PS: The logs above are shown using a DSA key, I have since changed to RSA keys and it works fine and in the same way.

ksprashu
  • 111
  • 5
  • Unfortunately the server side is an external client that aren't very helpful. Since it works on Filezilla and sftp command-line, they aren't too interested in helping debug the script. Using only key or password on Paramiko doesn't work since as soon as I try to get the sftp session from the client, it terminates with "AuthenticationException" or "Connection is Dead". – ksprashu Feb 26 '21 at 02:50
  • I also quickly wrote up a script using Nodejs and `ssh2-sftp-client` library and that very easily was able to establish a connection and retrieve the directory listing. So there doesn't seem to be a problem with being able to script this, but something that either Paramiko is doing wrong, or some configuration that I may need to set up manually. – ksprashu Feb 26 '21 at 02:54
  • Wonderful, this absolutely worked. Thanks for pointing me to this Martin! – ksprashu Mar 01 '21 at 03:18
  • Absolutely yes. Did that, but my vote isn't reflecting (yet) due to reputation. Hopefully soon. – ksprashu Mar 01 '21 at 18:07

1 Answers1

0

The answer provided to the linked question worked beatifully for my use case.
Multi-factor authentication (password and key) with Paramiko

Thanks Martin Prikryl!

ksprashu
  • 111
  • 5