4

I've been working on a python project to send information to a server, and while I found some nice code to read the file and send it, I can only send the entire thing at once. I want instead to be able to read 1000 characters at a time, and loop until the file's end. I'm sure that this is pretty easy, but I'm not sure where to change what I've been doing to make it happen. This might also be due to unfamiliarity with the method of reading, but I am researching it and not finding any information that has clarified it for me. This is what I'm working with:

with open(localFileTT, 'rb', 0) as f: #open the file 
    for fstr in f:
        str.encode(fstr)
        print(fstr)
        if 'str' in fstr:
            break

I have not yet introduced network functions in this code, as it is only test material to figure out how to read only 1000 at a time. It certainly reads the entire section well enough, but I don't know how to change it so as to read only parts at a time. I would try just adding read(1000) but that prints emptiness! I'm not sure where exactly the reading is occurring here, and unfortunately the articles I've read online thus far on parts of this code (such as with) has not helped any. If anyone can point me in the right direction, then I will be quite grateful.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
user2828965
  • 405
  • 1
  • 6
  • 17
  • 1
    Note that `str` is immutable; `str.encode(fstr)` returns a **new** string object, but your code drops that by the wayside. `fstr` itself is not changed. – Martijn Pieters Oct 26 '13 at 16:51

1 Answers1

9

file.read() takes a size argument:

Read at most size bytes from the file (less if the read hits EOF before obtaining size bytes). If the size argument is negative or omitted, read all data until EOF is reached.

You can use that in a loop, combined with the iter() function (with the second argument set to '' as a sentinel), and functools.partial() for efficiency:

from functools import partial

with open(localFileTT, 'rb', 0) as f:
    for chunk in iter(partial(f.read, 1000), ''):
        # chunk is up to 1000 characters long

The alternative would be to use a while loop:

with open(localFileTT, 'rb', 0) as f:
    while True:
        chunk = f.read(1000)
        if not chunk:
            # EOF reached, end loop
            break
        # chunk is up to 1000 characters long

If you already tried read(1000) and got an empty string back, your file was already at EOF at the time.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • The alternative worked out for me. Now I just need to remember how to encode as byte array and send in packet. Thank you so much! – user2828965 Oct 26 '13 at 17:08
  • be careful to send a end-of-stream marker if you send chunks of data, or else the server won't know if the file is ended or connection is broken.And you shuld use 1024, 2048, 4096 ... not 1000, because some buffer mecanism(on socket, file IO, server-side) may work better with this values. – cox Oct 26 '13 at 18:22