1

Is there a better and cleaner way to stream data from a text file as it been written to in real time?

At the moment im using a server to simulate data being written continuously and using seek() method to only read the most recently added data and the print it to the screen.

The problem Im having is testing if the file is been written to or not and if its not then to sleep for a short period then test if its been written to then generate and output to print to the screen.

import time
seek1 = 198707
seek2 = 198719
file_open = open('C:\Users\Jay\Desktop\Tools\Uni-    Server\UniServerZ\core\\apache2\logs\\access.log')
#C:\Users\Jay\Desktop\Tools\Uni-Server\UniServerZ\core\apache2\logs
#C:\Users\Jay\PycharmProjects\Parsly\\access.log

def generate_list(read_lines):#properly build this with seek positional checks.
    #file_object.seek(seek_position)
    #lines = file_object.readlines()
    for i in read_lines:
        yield i


def seek_new_postion(generator):
    listt = []
    seek = file_open.tell()
    old_seek = None
    counter = 0
    while 1:
        if old_seek == seek and not counter == 5:
            print "sleeping"
            time.sleep(1)
            counter +=1
        else:
            p = generate_list(file_open.readlines())
            for i in p:
                print i
            time.sleep(1)
            print "___________________________"
            print "start seek: %s" % file_open.tell()
            print "old seek: %s" % old_seek
            print "___________________________"
            old_seek = seek
            seek = file_open.tell()
            counter = 0

print file_open.tell()
for i in seek_new_postion("test"):
print i
Jay Jamieson
  • 11
  • 2
  • You may get some inspiration from http://stackoverflow.com/a/1476006/2375207 – nicolallias Jun 25 '15 at 08:41
  • possible duplicate of [tail -f in python with no time.sleep](http://stackoverflow.com/questions/1475950/tail-f-in-python-with-no-time-sleep) – skyking Jun 25 '15 at 08:50
  • There are some problems with your solution as you don't update `old_seek` or `seek` if they happens to be the same - the loop will just sleep for five seconds. – skyking Jun 25 '15 at 08:52
  • @skyking i couldn't work out how to change the seek and old_seek so i used counter as a time out to try and read the file again from seek position. – Jay Jamieson Jun 25 '15 at 09:13
  • @Jay Jamieson Actually the `seek` and `tell` is quite pointless and overworked solution. You would have to save current position, `seek` to the end and use `tell` to get the length of the file and then to restore the file position again with `seek` to the saved position. Instead you could just rely on the fact that `readlines` will return empty result if there is nothing more to be read from the file (yet). – skyking Jun 25 '15 at 09:35
  • @skyking I see what you mean, the reason i wanted to use seek is to have a write out file when the program exits so that when it gets started it will start from the last seeked position – Jay Jamieson Jun 25 '15 at 09:51
  • @Jay Jamieson Then you'll do the same, but each time you read from it you either sleep (no lines returned) or get new position via `tell`. Then when you start next time you `seek` to the last position - a lot of telling, but not so much seeking. – skyking Jun 25 '15 at 10:20
  • Thanks, i found where i was going wrong, with the seek in part of the if statement. – Jay Jamieson Jun 25 '15 at 11:08

0 Answers0