Ansible: connection to AWS host fails with unknown SSH error

Question

I have an EC2 machine that I can SSH into from commandline. Now, I am trying to execute a playbook where this machine is the inventory. My inventory file looks like this:

default ansible_host=ec2-xx.xx.xx.xx.us-west-2.compute.amazonaws.com ansible_user='ubuntu' ansible_ssh_private_key_file='~/.ssh/my_aws.pem'

When I try to execute a playbook, ansible fails with:

ERROR! SSH encountered an unknown error

Generating verbose logging shows that "Authentication succeeded (publickey)". I am at a loss as to why the connection fails. Verbose logs below:

{"changed": false, "msg": "ERROR! SSH encountered an unknown error. The output was:
OpenSSH_6.9p1, LibreSSL 2.1.8
debug1: Reading configuration data /Users/my_user/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket \"/Users/my_user/.ansible/cp/ansible-ssh-ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com-22-ubu\" does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com [xx.xxx.x.xx] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: fd 3 clearing O_NONBLOCK
debug1: Connection established.
debug3: timeout: 9962 ms remain after connect
debug1: key_load_public: No such file or directory
debug1: identity file /Users/my_user/.ssh/my_aws.pem type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/my_user/.ssh/my_aws.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 pat OpenSSH_6.6.1* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com:22 as 'ubuntu'
debug3: hostkeys_foreach: reading file \"/Users/my_user/.ssh/known_hosts\"
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: zlib@openssh.com,zlib,none
debug2: kex_parse_kexinit: zlib@openssh.com,zlib,none
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: first_kex_follows 0 
debug2: kex_parse_kexinit: reserved 0 
debug2: kex_parse_kexinit: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: first_kex_follows 0 
debug2: kex_parse_kexinit: reserved 0 
debug1: kex: server->client chacha20-poly1305@openssh.com <implicit> zlib@openssh.com
debug1: kex: client->server chacha20-poly1305@openssh.com <implicit> zlib@openssh.com
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:xxxxxxxxxxx
debug3: hostkeys_foreach: reading file \"/Users/my_user/.ssh/known_hosts\"
debug3: hostkeys_foreach: reading file \"/Users/my_user/.ssh/known_hosts\"
Warning: Permanently added 'ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com,xx.xxx.x.xx' (ECDSA) to the list of known hosts.
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /Users/my_user/.ssh/my_aws.pem (0x0), explicit
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred gssapi-with-mic,gssapi-keyex,hostbased,publickey
debug3: authmethod_lookup publickey
debug3: remaining preferred: ,gssapi-keyex,hostbased,publickey
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/my_user/.ssh/my_aws.pem
debug3: sign_and_send_pubkey: RSA SHA256:xxxxxxxxxxxx
debug2: we sent a publickey packet, wait for reply
debug1: Enabling compression at level 6.
debug1: Authentication succeeded (publickey).
Authenticated to ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com ([xx.xxx.x.xx]:22).
debug1: setting up multiplex master socket
debug3: muxserver_listen: temporary control path /Users/my_user/.ansible/cp/ansible-ssh-ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com-22-ubuntu.JNNIIMYaFyD7UIF3
unix_listener: \"/Users/my_user/.ansible/cp/ansible-ssh-ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com-22-ubuntu.JNNIIMYaFyD7UIF3\" too long for Unix domain socket
", "unreachable": true}

score 3 · Accepted Answer · edited May 23 '17 at 12:31

3

unix_listener: \"/Users/my_user/.ansible/cp/ansible-ssh-ec2-xx-xxx-x-xx.us-west-2.compute.amazonaws.com-22-ubuntu.JNNIIMYaFyD7UIF3\" too long for Unix domain socket

is the problem. You need to configure Ansible to store shorter UNIX domain sockets. This path shouldn't be longer then 92 characters, as unix(7) proposes:

When coding portable applications, keep in mind that some implementations have sun_path as short as 92 bytes.

One possibility is to set in your ansible.cfg shorter control_path, for example:

control_path = %(directory)s/%%C

Or disable multiplexing completely (will have impact on performance!):

ssh_args = -o ControlMaster=off

The options are described in documentation and I answered similar question some time ago.

edited May 23 '17 at 12:31

Community

1
1

answered Feb 24 '16 at 08:11

Jakuje

24,773
12
69
75

Thanks Jakuke. This is a newbie question, but ansible config requires a section name. Does this do in [defaults]? If not, what is the right section? – Raj Feb 24 '16 at 17:28
Checking the linked documentation mentions these options under `[ssh_connection]` section. – Jakuje Feb 24 '16 at 17:36
Thanks. I tried that, and now I have a different error. This is what I see in the tail end of my file: `debug1: Authentication succeeded (publickey). Authenticated to ec2-xx-xxx-xx-xxx.us-west-2.compute.amazonaws.com ([xx.xxx.xx.xxx]:22). debug1: setting up multiplex master socket debug3: muxserver_listen: temporary control path ./s/ec2-xx-xxx-xx-xxx.us-west-2.compute.amazonaws.com-ubuntu.oDzqobFGyqbm992H bind: No such file or directory unix_listener: cannot bind to path: ./s/ec2-xx-xxx-xx-xxx.us-west-2.compute.amazonaws.com-ubuntu.oDzqobFGyqbm992H` – Raj Feb 24 '16 at 17:41
1

Figured it out. Based on this thread: https://github.com/ansible/ansible/issues/11536 it should have been `control_path = %(directory)s/%%C`. It was the static path starting with `.s/...` that was causing problems. – Raj Feb 24 '16 at 17:47

Ansible: connection to AWS host fails with unknown SSH error

1 Answers1