41

Started Python a week ago and I have some questions to ask about reading and writing to the same files. I've gone through some tutorials online but I am still confused about it. I can understand simple read and write files.

openFile = open("filepath", "r")
readFile = openFile.read()
print readFile 

openFile = open("filepath", "a")
appendFile = openFile.write("\nTest 123")

openFile.close()

But, if I try the following I get a bunch of unknown text in the text file I am writing to. Can anyone explain why I am getting such errors and why I cannot use the same openFile object the way shown below.

# I get an error when I use the codes below:       
openFile = open("filepath", "r+")
writeFile = openFile.write("Test abc")

readFile = openFile.read()
print readFile

openFile.close()

I will try to clarify my problems. In the example above, openFile is the object used to open file. I have no problems if I want write to it the first time. If I want to use the same openFile to read files or append something to it. It doesn't happen or an error is given. I have to declare the same/different open file object before I can perform another read/write action to the same file. 

#I have no problems if I do this:    
openFile = open("filepath", "r+")
writeFile = openFile.write("Test abc")

openFile2 = open("filepath", "r+")
readFile = openFile2.read()
print readFile

openFile.close()

I will be grateful if anyone can tell me what I did wrong here or is it just a Pythong thing. I am using Python 2.7. Thanks!

Cryssie
  • 3,047
  • 10
  • 54
  • 81

4 Answers4

32

Updated Response:

This seems like a bug specific to Windows - http://bugs.python.org/issue1521491.

Quoting from the workaround explained at http://mail.python.org/pipermail/python-bugs-list/2005-August/029886.html

the effect of mixing reads with writes on a file open for update is entirely undefined unless a file-positioning operation occurs between them (for example, a seek()). I can't guess what you expect to happen, but seems most likely that what you intend could be obtained reliably by inserting

fp.seek(fp.tell())

between read() and your write().

My original response demonstrates how reading/writing on the same file opened for appending works. It is apparently not true if you are using Windows.

Original Response:

In 'r+' mode, using write method will write the string object to the file based on where the pointer is. In your case, it will append the string "Test abc" to the start of the file. See an example below:

>>> f=open("a","r+")
>>> f.read()
'Test abc\nfasdfafasdfa\nsdfgsd\n'
>>> f.write("foooooooooooooo")
>>> f.close()
>>> f=open("a","r+")
>>> f.read()
'Test abc\nfasdfafasdfa\nsdfgsd\nfoooooooooooooo'

The string "foooooooooooooo" got appended at the end of the file since the pointer was already at the end of the file.

Are you on a system that differentiates between binary and text files? You might want to use 'rb+' as a mode in that case.

Append 'b' to the mode to open the file in binary mode, on systems that differentiate between binary and text files; on systems that don’t have this distinction, adding the 'b' has no effect. http://docs.python.org/2/library/functions.html#open

Community
  • 1
  • 1
siddharthlatest
  • 2,237
  • 1
  • 20
  • 24
  • That's the weird part. It seems that I get an error if I tried these lines: f=open("C:\Users\MooMoo\Desktop\clothes.txt","r+") print f.read() f.write("foooooooooooooo") . The error messages I get: Traceback (most recent call last): File "C:\Users\MooMoo\Desktop\Python27\PyProject\src\test.py", line 43, in f.write("foooooooooooooo") IOError: [Errno 0] Error f.close() – Cryssie Jan 11 '13 at 05:20
  • This seems like a Windows specific issue then. I am updating my response. Gist: Use f.seek(f.tell()) before writing. – siddharthlatest Jan 12 '13 at 07:18
12

Every open file has an implicit pointer which indicates where data will be read and written. Normally this defaults to the start of the file, but if you use a mode of a (append) then it defaults to the end of the file. It's also worth noting that the w mode will truncate your file (i.e. delete all the contents) even if you add + to the mode.

Whenever you read or write N characters, the read/write pointer will move forward that amount within the file. I find it helps to think of this like an old cassette tape, if you remember those. So, if you executed the following code:

fd = open("testfile.txt", "w+")
fd.write("This is a test file.\n")
fd.close()

fd = open("testfile.txt", "r+")
print fd.read(4)
fd.write(" IS")
fd.close()

... It should end up printing This and then leaving the file content as This IS a test file.. This is because the initial read(4) returns the first 4 characters of the file, because the pointer is at the start of the file. It leaves the pointer at the space character just after This, so the following write(" IS") overwrites the next three characters with a space (the same as is already there) followed by IS, replacing the existing is.

You can use the seek() method of the file to jump to a specific point. After the example above, if you executed the following:

fd = open("testfile.txt", "r+")
fd.seek(10)
fd.write("TEST")
fd.close()

... Then you'll find that the file now contains This IS a TEST file..

All this applies on Unix systems, and you can test those examples to make sure. However, I've had problems mixing read() and write() on Windows systems. For example, when I execute that first example on my Windows machine then it correctly prints This, but when I check the file afterwards the write() has been completely ignored. However, the second example (using seek()) seems to work fine on Windows.

In summary, if you want to read/write from the middle of a file in Windows I'd suggest always using an explicit seek() instead of relying on the position of the read/write pointer. If you're doing only reads or only writes then it's pretty safe.

One final point - if you're specifying paths on Windows as literal strings, remember to escape your backslashes:

fd = open("C:\\Users\\johndoe\\Desktop\\testfile.txt", "r+")

Or you can use raw strings by putting an r at the start:

fd = open(r"C:\Users\johndoe\Desktop\testfile.txt", "r+")

Or the most portable option is to use os.path.join():

fd = open(os.path.join("C:\\", "Users", "johndoe", "Desktop", "testfile.txt"), "r+")

You can find more information about file IO in the official Python docs.

Cartroo
  • 4,233
  • 20
  • 22
  • Hey Cartroo, unfortunately, it seems I do understand your codes but somehow when I compile, it doesn't give me what you've shown me. For example, I followed the exact set of codes you've given in your first set of examples. When I check the file, the text in the file only shows "This is a test file" and not "This IS a test file". It looks like write didn't work for me. I am using Python 2.7 if it helps. – Cryssie Jan 11 '13 at 11:24
  • 1
    It looks from your comments like you're using Windows - as I mentioned in my reply, mixing `read()` and `write()` in the same file doesn't seem to work very well under Windows, even when using binary mode - I'm afraid I'm not familiar enough with Windows to say why. However, `seek()` seems to work so add this line `fd.seek(4)` just *before* the line `fd.write(" IS")`. That explicit `seek()` seems to get things to work on Windows. If I can track down information on why the original version doesn't work on Windows, I'll post another comment here. – Cartroo Jan 11 '13 at 13:44
  • 1
    I was so curious about this specific behaviour that I submitted a new question about it: http://stackoverflow.com/questions/14279658/mixing-read-and-write-on-python-files-in-windows – Cartroo Jan 11 '13 at 13:58
  • Actually, it doesn't update anything if you used it to read or write after declaring the "fileOpen" object the first time. To write to the same file, I need to declare another "fileOpen" object. I am using Windows 7 and using Python with Eclipse. I am still learning Python and almost all tutorials or books says that this shouldn't be happening so I am very confused. – Cryssie Jan 11 '13 at 14:21
3

Reading and Writing happens where the current file pointer is and it advances with each read/write. In your particular case, writing to the openFile, causes the file-pointer to point to the end of file. Trying to read from the end would result EOF. You need to reset the file pointer, to point to the beginning of the file before through seek(0) before reading from it

Abhijit
  • 62,056
  • 18
  • 131
  • 204
0

You can read, modify and save to the same file in python but you have actually to replace the whole content in file, and to call before updating file content:

# set the pointer to the beginning of the file in order to rewrite the content
edit_file.seek(0)

I needed a function to go through all subdirectories of folder and edit content of the files based on some criteria, if it helps:

new_file_content = ""
for directories, subdirectories, files in os.walk(folder_path):
    for file_name in files:
        file_path = os.path.join(directories, file_name)
        # open file for reading and writing
        with io.open(file_path, "r+", encoding="utf-8") as edit_file:
            for current_line in edit_file:
                if condition in current_line:
                    # update current line
                    current_line = current_line.replace('john', 'jack')
                new_file_content += current_line
            # set the pointer to the beginning of the file in order to rewrite the content
            edit_file.seek(0)
            # delete actual file content
            edit_file.truncate()
            # rewrite updated file content
            edit_file.write(new_file_content)
            # empties new content in order to set for next iteration
            new_file_content = ""
            edit_file.close()
adambogdan1993
  • 146
  • 4