Ansible Windows Deployment - 'Connection aborted.', error(104, 'Connection reset by peer')

Question

I am facing an issue with deployment on Windows 2008 server using Ansible. I have a jenkins pipeline which has an Ansible Deployment stage wherein I drop the files on to the target servers from inventory. I have been able to do this successfully for quite a few servers. However for some servers, I see the above error message.

Here is the output from jenkins pipeline

TASK [debug] *******************************************************************
ok: [ch3dw026394.mydomain.com] => {
    "msg": "Hello world!"
}

TASK [Testing connection to windows] *******************************************
fatal: [ch3dw026394.mydomain.com]: UNREACHABLE! => {"changed": false, "msg": "ntlm: ('Connection aborted.', error(104, 'Connection reset by peer'))", "unreachable": true}

Here is the Ansible playbook module for win_ping

  #Testing connection with Target
  - debug: 
    msg: "Testing connection to windows {{ inventory_hostname }}"

  - name: Testing connection to windows
    win_ping: null
    any_errors_fatal: true

I did quite a bit of research on this and none of the following options worked

Turned off UAC on the Windows 2008 server.
Did not work Some posts online said that this error happens because the inventory file does not contain FQDN, but contains IP. My inventory file contains FQDN (Fully Qualified domain name) and still does not work Another post said that the port number(5986) in the vars.yml should be commented out and instead have hostname:5986 in the inventory file. Still did not work
I tried deleting the listeners on the server and ran the powershell script to enable WinRM (https://github.com/ansible/ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1). This also did not work. Deleted the listeners using
```
winrm delete winrm/config/Listener?Address=*+Transport=HTTP
winrm delete winrm/config/Listener?Address=*+Transport=HTTS
```
Set WinRM to have basic Auth = true and also AllowUncrypted = true (see below) and still did not work. (Please note I did not have to do any of this in my other Windows 2008 servers and it worked absolutely fine).
```
winrm set winrm/config/service '@{AllowUnencrypted= "true"}'
Set-Item -Path "WSMan:\localhost\Service\Auth\Basic" -Value $true
```

Please can you help me get past this error.

Thanks in advance for your assistance! Warm Regards

score 2 · Answer 1 · edited Feb 18 '19 at 16:58

2

i had similar problem when trying to ping-pong the windows 10 node:

examplenode | UNREACHABLE! => {
    "changed": false, 
    "msg": "ssl: ('Connection aborted.', error(104, 'Connection reset by peer'))", 
    "unreachable": true

re issuing certificate helped with option -ForceNewSSLCert on ConfigureRemotingforAnsible.ps1 script. (on target node)

this happened to two nodes in my collection, what i noticed is that both nodes were on 1607 version of windows 10

( if someone wants to track down the problem)

edited Feb 18 '19 at 16:58

Sanpas

1,170
10
29

answered Feb 18 '19 at 14:13

nikola

31
2

This one worked for me on a Windows 10 node following the 2004 update. You can find the script here: https://github.com/ansible/ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1 – Teddy Sep 23 '20 at 14:00
1

Running `winrm quickconfig -transport:https` even tells me why: "Cannot create a WinRM listener on HTTPS because this machine does not have an appropriate certificate. To be used for SSL, a certificate must have a CN matching the hostname, be appropriate for Server Authentication, and not be expired, revoked, or self-signed.". Which in my case is due to `CN = packer`, so hashicorp leaves WinRM in a broken state after using it. – azrdev Dec 08 '22 at 10:16
this is helpful info, with enough persistance, e.g. stopping, starting windown remote management in services, i finally got the winrm quckconfig -transport:https to say the same. before that it didnt give that message. I can say i was able to get it to work, with `CN=ansible`, but the ansible botstrap script keeps saying ` New-Item -Path WSMan:\localhost\ClientCertificate `. I see the port is open though, so my issue may be different. This is one of the hardest things to get to work! – blamb Jan 25 '23 at 19:42

score 1 · Answer 2 · answered Jan 25 '19 at 22:16

Ok Folks, for those who are wondering if this was fixed or not, the answer is Yes, I got it working. However, whether or not this is the correct approach is something which I am not sure about.

So here is the workaround as to how to get around this error:

The issue was due to TLS 1.2 registry setting.

So go to the registry editor (regedit) and traverse to the following location HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols
Check if there is an entry for TLS 1.2. If yes, then delete it. Now try deploying again, it should work!

As i said, this is a work around. If TLS 1.2 needs to be present, then a deeper dive needs to be done from Ansible playbook perspective.

score 0 · Answer 3 · answered Feb 13 '20 at 22:10

When running with -vvv, I got a detailed stacktrace from Ansible. The openstack: on every line is added by packer.

    openstack: TASK [Disable check for non-interactive shells, all shells interactive from now on] ***
    openstack: task path: image-generator/provisioning-scripts/ansible/win_paths.yaml:7
    openstack: Using module file /home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/ansible/modules/windows/win_lineinfile.ps1
    openstack: Pipelining is enabled.
    openstack: <xxx> ESTABLISH WINRM CONNECTION FOR USER: Administrator on PORT 5986 TO xxx
    openstack: EXEC (via pipeline wrapper)
    openstack: The full traceback is:
    openstack: Traceback (most recent call last):
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    openstack:     chunked=chunked,
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    openstack:     six.raise_from(e, None)
    openstack:   File "<string>", line 3, in raise_from
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    openstack:     httplib_response = conn.getresponse()
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/http/client.py", line 1344, in getresponse
    openstack:     response.begin()
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/http/client.py", line 306, in begin
    openstack:     version, status, reason = self._read_status()
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/http/client.py", line 267, in _read_status
    openstack:     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/socket.py", line 589, in readinto
    openstack:     return self._sock.recv_into(b)
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/ssl.py", line 1071, in recv_into
    openstack:     return self.read(nbytes, buffer)
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/ssl.py", line 929, in read
    openstack:     return self._sslobj.read(len, buffer)
    openstack: ConnectionResetError: [Errno 104] Connection reset by peer
    openstack:
    openstack: During handling of the above exception, another exception occurred:
    openstack:
    openstack: Traceback (most recent call last):
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    openstack:     timeout=timeout
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    openstack:     method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/util/retry.py", line 400, in increment
    openstack:     raise six.reraise(type(error), error, _stacktrace)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
    openstack:     raise value.with_traceback(tb)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    openstack:     chunked=chunked,
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    openstack:     six.raise_from(e, None)
    openstack:   File "<string>", line 3, in raise_from
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    openstack:     httplib_response = conn.getresponse()
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/http/client.py", line 1344, in getresponse
    openstack:     response.begin()
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/http/client.py", line 306, in begin
    openstack:     version, status, reason = self._read_status()
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/http/client.py", line 267, in _read_status
    openstack:     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/socket.py", line 589, in readinto
    openstack:     return self._sock.recv_into(b)
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/ssl.py", line 1071, in recv_into
    openstack:     return self.read(nbytes, buffer)
    openstack:   File "/nix/store/k5rdcbcwwpvj7l9f1yvd5mfggcfz16kk-python3-3.7.5/lib/python3.7/ssl.py", line 929, in read
    openstack:     return self._sslobj.read(len, buffer)
    openstack: urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
    openstack:
    openstack: During handling of the above exception, another exception occurred:
    openstack:
    openstack: Traceback (most recent call last):
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/ansible/executor/task_executor.py", line 146, in run
    openstack:     res = self._execute()
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/ansible/executor/task_executor.py", line 645, in _execute
    openstack:     result = self._handler.run(task_vars=variables)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/ansible/plugins/action/normal.py", line 46, in run
    openstack:     result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/ansible/plugins/action/__init__.py", line 923, in _execute_module
    openstack:     res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/ansible/plugins/action/__init__.py", line 1071, in _low_level_execute_command
    openstack:     rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/ansible/plugins/connection/winrm.py", line 548, in exec_command
    openstack:     result = self._winrm_exec(cmd_parts[0], cmd_parts[1:], from_exec=True, stdin_iterator=stdin_iterator)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/ansible/plugins/connection/winrm.py", line 475, in _winrm_exec
    openstack:     resptuple = self.protocol.get_command_output(self.shell_id, command_id)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/winrm/protocol.py", line 449, in get_command_output
    openstack:     self._raw_get_command_output(shell_id, command_id)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/winrm/protocol.py", line 468, in _raw_get_command_output
    openstack:     res = self.send_message(xmltodict.unparse(req))
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/winrm/protocol.py", line 243, in send_message
    openstack:     resp = self.transport.send_message(message)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/winrm/transport.py", line 323, in send_message
    openstack:     response = self._send_message_request(prepared_request, message)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/winrm/transport.py", line 328, in _send_message_request
    openstack:     response = self.session.send(prepared_request, timeout=self.read_timeout_sec)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    openstack:     r = adapter.send(request, **kwargs)
    openstack:   File "/home/jdanek/.local/share/virtualenvs/image-generator-cHKWQna6/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    openstack:     raise ConnectionError(err, request=request)
    openstack: requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
    openstack: fatal: [xxx]: FAILED! => {
    openstack:     "msg": "Unexpected failure during module execution.",
    openstack:     "stdout": ""
    openstack: }

I've resolved this error by adding the following to Ansible command line

--extra_vars 'ansible_shell_type=powershell ansible_shell_executable=None'

I added ansible_shell_type=powershell because Ansible documentation says this should be present (with value of =powershell or =cmd). I saw the ansible_shell_executable=None on some Packer with Ansible issues. It probably does not hurt.

In addition to these, I was already using the extra var ansible_winrm_server_cert_validation=ignore.

Other advice I found was to reinstall ansible in a new Python virtual environment, to ensure the requests package is up to date. I did not try since the above already ran fine.

I also experimented with extra var ansible_winrm_transport=basic and kdestroy-ing my kerberos tickets for other systems that I held.

score 0 · Answer 4 · answered Sep 16 '22 at 17:39

What resolved it for me:

The actual error was TLS 1.3 was set enabled (1), in registry, but was not set DisabledByDefault. setting that to 1 resolved the long looming ansible ssl connection error.

Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.3\Server

If that doesnt work, you can delete all keys from SCHANNEL\protocols which is default on some windows systems, and try that.

It looks like ansible was being forced to use TLS1.3 where the python install / urllib3 doesnt seem to be supporting that for the ssl handshake.

ansible_connection: winrm
ansible_winrm_scheme: https
ansible_port: 5986 #didnt break it, leaving this enabled
ansible_winrm_cert_pem: ..
ansible_winrm_cert_key_pem: ...
ansible_winrm_transport: certificate
ansible_winrm_server_cert_validation: ignore # maybe because i WAS USING IP, not hostname, also the cert  CN is "ansible" anyway, not matching any hostname.

https://community.tenable.com/s/article/How-to-view-and-change-the-Windows-Registry-Settings-for-the-SSL-TLS-Protocols-on-a-Windows-Host

Ansible Windows Deployment - 'Connection aborted.', error(104, 'Connection reset by peer')

4 Answers4