I'm having a problem handling socket timeouts with python35 ftplib. When a socket timeout error occurs, for some reason I am unable to catch the exception and the script raises the error anyway and exits. Here is the relevant code block:
try:
ftp = FTP(self.config.base_url, timeout=400)
ftp.login()
ftp.cwd(self.config.root_path)
ftp.retrbinary("RETR {0}".format(os.path.join(self.root_path, file_path)), fp.write, 1024)
ftp.quit()
except socket.timeout:
self.download_file(file_path)
For some reason this script will still error out with a socket timeout exception, how is that possible? A general catch-all doesn't work either. Here is the stack trace of the error:
File "ftp.py", line 82, in __init__
self.ftp = FTP(self.config.base_url, timeout=400)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ftplib.py", line 118, in __init__
self.connect(host)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ftplib.py", line 156, in connect
self.welcome = self.getresp()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ftplib.py", line 235, in getresp
resp = self.getmultiline()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ftplib.py", line 221, in getmultiline
line = self.getline()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ftplib.py", line 203, in getline
line = self.file.readline(self.maxline + 1)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 576, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
As you can see, this is the socket.timeout error being thrown, so how is that not caught? I was unable to find any useful info on how to resolve this issue after many hours of internet research, any insight into this issue would be much appreciated.
For reference, here is the relevant code block of socket.py:
def readinto(self, b):
"""Read up to len(b) bytes into the writable buffer *b* and return
the number of bytes read. If the socket is non-blocking and no bytes
are available, None is returned.
If *b* is non-empty, a 0 return value indicates that the connection
was shutdown at the other end.
"""
self._checkClosed()
self._checkReadable()
if self._timeout_occurred:
raise OSError("cannot read from timed out object")
while True:
try:
return self._sock.recv_into(b)
except timeout:
self._timeout_occurred = True
raise
except error as e:
if e.args[0] in _blocking_errnos:
return None
raise
UPDATE: It seems the issue is that ftplib behaves unexpectedly if the timeout passed to the FTP constructor is greater than the default for a socket timeout. At the time of this edit there is an open python issue to address this behavior: http://bugs.python.org/issue30956
