Paramiko - sftp client used to get last line of log file

Question

I'd like to use paramiko and an sftp client to poll a log file for the last line. I'm aware of sshtail module in python, but using it goes against the coding standards where I am right now.

I previously used that and now I'm wondering how to go about reading the last line of a log file?

Thanks,

Parth

EDIT2:

try: 
    self.logger.info("SSH Log: trying to connect to: " + self.ssh_server_ip + "," + str(self.ssh_port) + "," + self.ssh_username + "," + self.ssh_password) 
    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(self.ssh_server_ip, self.log_server_port, self.ssh_username, self.ssh_password)
    self.logger.info("SSH LOG: Deleting files from HTTP Upload Server")
    sftp = client.open_sftp()
    remote_command = "tail -n1 /var/log/apache2/access.log"
    stdin, stdout, stderr = client.exec_command(remote_command)
    last_line = stdout.read()
    old_line = last_line
    while 1:
        remote_command = "tail -n1 /var/log/apache2/access.log"
        stdin, stdout, stderr = client.exec_command(remote_command)
        last_line = stdout.read()
        if last_line != old_line:
            finish_line = last_line
            break
    self.logger.info("SSH Log: closing connection")
    sftp.close()
    client.close()
    except Exception, e:
        self.logger.error(str(e))
        self.logger.error("Failed to delete file on HTTP server: " + str(e))
    except:
        self.logger.error("Failed to delete file on HTTP server")

I'm assuming the file is on a remote server then? What have you tried so far? — Ben, Mar 24 '12 at 21:30
It's on a remote linux server running Ubuntu 11.10. I've just used the sshtail module to tail the file. I should only expect a non blank log. Once I get it, I save it and parse it. — Parth, Mar 24 '12 at 21:32
I meant do you have any code to post, which is your attempt to solve the problem? It's expected that you make some attempt first, rather than just asking for code: [ask] — Ben, Mar 24 '12 at 21:36
possible duplicate of [Python to emulate remote tail -f?](http://stackoverflow.com/questions/7680055/python-to-emulate-remote-tail-f) — Ben, Mar 24 '12 at 21:37
I don't think it is, because i must use SFTP client in paramiko — Parth, Mar 26 '12 at 15:36

Preet Kukreti · Accepted Answer · 2012-03-26T16:24:09.997

2

It can be faster and easier to just call tail -n1 on the file remotely via the shell and read its stdout. Something like:

remote_command = "tail -n1 /var/log/apache2/access.log"
stdin, stdout, stderr = client.exec_command(remote_command)
last_line = stdout.read()

edited Mar 26 '12 at 16:24

answered Mar 26 '12 at 16:18

Preet Kukreti

8,417
28
36

Should I just do a tail -f actually? Can you please provide code that does what you're asking? – Parth Mar 26 '12 at 16:21
`tail -f` is *follow* mode - probably not what you want. I've updated my response with some code. – Preet Kukreti Mar 26 '12 at 16:28
-n1 would get the last line of the file. I'm waiting for a file to download and the access log would indicate that it's finished downloading. So how would that work with tail -f – Parth Mar 26 '12 at 16:30
ahh. You could poll it with `-n1` (just put it inside a loop), but it can have a lot of overhead. Im not quite sure you will be able to do a check inside stdout.read() because that is a blocking call that takes no options for testing/waiting AFAIK. – Preet Kukreti Mar 26 '12 at 17:17
If I poll it with -n1 in a loop, won't it read the last line already written to the file? ideally i'd like to poll the file for a new line, obtain the immediate line and parse that later on – Parth Mar 26 '12 at 17:34
what do you mean by 'immediate line'? I thought you were trying to read the last line written to file. Now that I look at it, your while loop is in some infinite loop? Is it expected to throw an exception? Where is the delete actually happening in your code? – Preet Kukreti Mar 26 '12 at 19:09
I ended up using a variation of your code which i have posted in my original post. Thanks for your help – Parth Mar 26 '12 at 20:58

Paramiko - sftp client used to get last line of log file

1 Answers1