How to avoid ConnectionResetError?

Question

OS: Windows 10 x64

Python 3.8.0

async def main():
    rss_urls = []
    for item in db[config['mongodb']['channels_collection']].find({'url': {'$ne': 'No RSS'}}):
        rss_urls.append(item['url'])

    while rss_urls:
        task_urls = []
        for _ in range(tasks_limit):
            try:
                task_urls.append(rss_urls.pop(0))
            except IndexError:
                break

        tasks = [asyncio.create_task(rss_downloader(rss)) for rss in task_urls]
        await asyncio.gather(*tasks)

        with open(f'{work_dir}/not_found.txt', 'a') as file:
            for rss in not_found_rss:
                file.write(f'{rss}\n')
    print(True)


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

Sometimes I get the following error:

Traceback (most recent call last):
  File "C:\Python3\lib\asyncio\windows_events.py", line 453, in finish_recv
    return ov.getresult()
OSError: [WinError 64] The specified network name is no longer available

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "rss_parser.py", line 114, in <module>
    loop.run_until_complete(main())
  File "C:\Python3\lib\asyncio\base_events.py", line 608, in run_until_complete
    return future.result()
  File "rss_parser.py", line 104, in main
    await asyncio.gather(*tasks)
  File "rss_parser.py", line 87, in rss_downloader
    logging.exception(f'{rss}')
  File "C:\Python3\lib\site-packages\httpx\client.py", line 884, in __aexit__
    await self.close()
  File "C:\Python3\lib\site-packages\httpx\client.py", line 873, in close
    await self.dispatch.close()
  File "C:\Python3\lib\site-packages\httpx\dispatch\connection_pool.py", line 179, in close
    await connection.close()
  File "C:\Python3\lib\site-packages\httpx\dispatch\connection.py", line 173, in close
    await self.h11_connection.close()
  File "C:\Python3\lib\site-packages\httpx\dispatch\http11.py", line 68, in close
    await self.socket.close()
  File "C:\Python3\lib\site-packages\httpx\concurrency\asyncio.py", line 207, in close
    await self.stream_writer.wait_closed()
  File "C:\Python3\lib\asyncio\streams.py", line 376, in wait_closed
    await self._protocol._get_close_waiter(self)
  File "C:\Python3\lib\asyncio\proactor_events.py", line 280, in _loop_reading
    data = fut.result()
  File "C:\Python3\lib\asyncio\windows_events.py", line 808, in _poll
    value = callback(transferred, key, ov)
  File "C:\Python3\lib\asyncio\windows_events.py", line 457, in finish_recv
    raise ConnectionResetError(*exc.args)
ConnectionResetError: [WinError 64] The specified network name is no longer available

Because of this error, the script stops working, how can I fix it?

I tried to run this script on Linux Subsystems with Ubuntu installed and after several runs of the script this error does not exist. But still, I would like to know what it is and how can I fix it for Windows too?

score 0 · Accepted Answer · answered Dec 13 '19 at 19:45

ConnectionResetError seems to be an issue on server side, it can happen arbitrarily from time to time.

You can't fix it from client, but you can suppress the exception (to stop script from shutting down) and optionally retry failed operation (since it can succeed next time).

To suppress the exception do something like this:

async def my_rss_downloader(rss):
    try:
        return await rss_downloader(rss)
    except ConnectionResetError:
        return None  # or other empty result like []


# later, in your code
tasks = [asyncio.create_task(my_rss_downloader(rss)) for rss in task_urls]

To retry take a look at Tenacity library.

How to avoid ConnectionResetError?

1 Answers1