Python: Override method in standard library

Question

I had some trouble with the ftplib module in Python 3, after some debugging, I found the mistake: the getline() method of the FTP class returns b'somestring' (but as a string, not bytes) instead of somestring. I can solve this with .decode("utf-8"), replacing the first line in the function

line = self.file.readline(self.maxline + 1)

with

line = self.file.readline(self.maxline + 1).decode("utf-8")

solves the error. But now, I want not to edit the file ftplib.py manually, instead I want to override it in my code. But while I'm using the FTP_TLS class, which inherits from FTP, I can't figure it out, how to do...

class FTP:
    def getline(self):
        line = self.file.readline(self.maxline + 1).decode("utf-8")
        ...

at the beginning in my code does not work, because FTP_TLS doesn't recognize the changes I made to FTP.

Sample:

import ftplib
import socket
import ssl


class FTP(ftplib.FTP):
    def getline(self):
        line = self.file.readline(self.maxline + 1).decode("utf-8")
        if len(line) > self.maxline:
            raise ftplib.Error("got more than %d bytes" % self.maxline)
        if self.debugging > 1:
            print('*get*', self.sanitize(line))
        if not line:
            raise EOFError
        if line[-2:] == ftplib.CRLF:
            line = line[:-2]
        elif line[-1:] in ftplib.CRLF:
            line = line[:-1]
        return line


class FTPS(ftplib.FTP_TLS):
    def __init__(self, host='', user='', passwd='', acct='', keyfile=None, certfile=None, timeout=60):
        ftplib.FTP_TLS.__init__(self, host=host, user=user, passwd=passwd, acct=acct, keyfile=keyfile,
                                certfile=certfile, timeout=timeout)

    def connect(self, host='', port=0, timeout=-999, source_address=None):
        if host != '':
            self.host = host
        if port > 0:
            self.port = port
        if timeout != -999:
            self.timeout = timeout
        try:
            self.sock = socket.create_connection((self.host, self.port), self.timeout)
            self.af = self.sock.family
            self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ssl_version=ssl.PROTOCOL_TLSv1_2)
            self.file = self.sock.makefile('rb')
            self.welcome = self.getresp()
        except Exception as e:
            print(e)
        return self.welcome

if __name__ == "__main__":
    ftps = FTPS()
    ftps.connect("host", 990)  # Returns b'welcomemessage'
    ftps.login("user", "pwd")
    ftps.prot_p()

    ftps.close()

Could you give a [mcve] of your current code and a better problem description than *"seems not to work"*? — jonrsharpe, Apr 24 '16 at 14:37
@jonrsharpe Updated the post, would be nice if you may have a look on my code! — linusg, Apr 24 '16 at 14:45
I feel like **minimal** needs reinforcing, and the problem description still doesn't tell us anything useful. — jonrsharpe, Apr 24 '16 at 14:45
@jonrsharpe Sorry, but I need to patch some ssl errors in the default `FTP_TLS` class... Can it be even more minimal? — linusg, Apr 24 '16 at 14:47
I can't tell you that - you need to strip it down as far as you can while still getting the same issue (in doing which you might solve the problem yourself...) Are the SSL issues actually related *to this specific problem*? — jonrsharpe, Apr 24 '16 at 14:48
@jonrsharpe But with removing the patch and own implementation of SSL FTP, there would be no sample class which inherits from `FTP`, so I would be able to override as I've done in my example! — linusg, Apr 24 '16 at 14:50
But your custom `FTP` subclass isn't actually getting *used* anywhere. Were you expecting that its `getline` implementation would somehow end up accessible to `FTPS`? It's still using the library's version of `FTP_TLS`, which uses the library's version of `FTP`. — jonrsharpe, Apr 24 '16 at 14:53
@linusg I cannot see any patches. You defined a subclass of `ftplib.FTP` and it seems you expect `ftplib.FTP_TLS` to know about it? Why? — Stop harming Monica, Apr 24 '16 at 14:53
@Goyo See http://stackoverflow.com/questions/12164470/python-ftp-implicit-tls-connection-issue, specially the accepted answer! — linusg, Apr 24 '16 at 14:55
@linusg that question doesn't appear to *have* an accepted answer. It also doesn't explain why you expected the library to magically find your `FTP` subclass. — jonrsharpe, Apr 24 '16 at 14:56
Also `FTP.getline()` returnig `bytes` instead of `string` is probably not an error but by design. It is not even part of the public API (it is undocumented) so whatever issue you want to fix I don't think that's the correct place to do it. — Stop harming Monica, Apr 24 '16 at 14:56
@linusg The question and answers you linked do not explain why you expect `ftplib.FTP_TLS` to know about your custom `FTP` class or why you want to change the behaviour of `FTP.getline()`. — Stop harming Monica, Apr 24 '16 at 15:05
@Goyo Ok guys, sorry for this a little bad described question. I've asked a more basic question here: http://stackoverflow.com/questions/36824914/override-method-of-class-in-another-file — linusg, Apr 24 '16 at 15:13
@jonrsharpe Ok guys, sorry for this a little bad described question. I've asked a more basic question here: http://stackoverflow.com/questions/36824914/override-method-of-class-in-another-file — linusg, Apr 24 '16 at 15:13
This still looks a lot like an [XY problem](http://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). You shouldn't have to monkey patch standard library classes in the first place. It's not clear from your question why `getline()` returning bytes should be a problem. — Lukas Graf, Apr 24 '16 at 15:27

score 2 · Answer 1 · answered Apr 24 '16 at 14:59

2

Firstly, it's not a flaw in ftplib, which does the right thing by not assuming an encoding for the byte stream. Repeat after me: bytes from a file or a socket are not strings until you decode them! Further reading:

http://www.joelonsoftware.com/articles/Unicode.html

Having said that, if you still want to monkeypatch ftplib to assume the encoding is always utf-8, you could potentially do it like this:

from ftplib import FTP

origGetLine = FTP.getline
def assumeUtf8GetLine(*args, **kwargs):
    return origGetLine(*args, **kwargs).decode('utf-8')
FTP.getline = assumeUtf8GetLine

answered Apr 24 '16 at 14:59

Lex Scarisbrick

1,540
1
24
31

Thanks, but does not work for me – linusg Apr 24 '16 at 15:02
3

@linusg *"does not work"* is a profoundly unhelpful error report – jonrsharpe Apr 24 '16 at 15:08
I tried to include it at the beginnign of my code, doesn't fix the error. – linusg Apr 24 '16 at 18:01
@linusg Including a complete `Traceback` would be helpful to isolate what's causing the error. Also, I posted my answer before I saw your edit with example code. One reason it may not work is that it `ftplib.FTP_TLS` inherits from `ftplib.FTP`, which the example code doesn't change. If you moved the `getline` method from your `FTP` subclass into your `FTPS` subclass it might work. Good luck! – Lex Scarisbrick Apr 24 '16 at 20:46
Thanks for your reply, the Problem is fixed now (See my last comment below the question). It was not an error, but the whole ftplib Module was confused by the trailing b' and ' caused by an incorrect conversion between Bytes and str. – linusg Apr 24 '16 at 23:30

Python: Override method in standard library

1 Answers1