Replace and overwrite instead of appending

Question

I have the following code:

import re
#open the xml file for reading:
file = open('path/test.xml','r+')
#convert to string:
data = file.read()
file.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>",data))
file.close()

where I'd like to replace the old content that's in the file with the new content. However, when I execute my code, the file "test.xml" is appended, i.e. I have the old content follwed by the new "replaced" content. What can I do in order to delete the old stuff and only keep the new?

http://stackoverflow.com/questions/2424000/read-and-overwrite-a-file-in-python — Kemal Fadillah, Jul 13 '12 at 11:06
When you say *"replace the old content that's in the file with the new content"*, you need to read in and transform the current contents `data = file.read()`. You don't mean "blindly overwrite it without needing to read it first". — smci, May 17 '18 at 08:25

score 161 · Accepted Answer · edited Sep 04 '19 at 03:10

161

You need seek to the beginning of the file before writing and then use file.truncate() if you want to do inplace replace:

import re

myfile = "path/test.xml"

with open(myfile, "r+") as f:
    data = f.read()
    f.seek(0)
    f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>", r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", data))
    f.truncate()

The other way is to read the file then open it again with open(myfile, 'w'):

with open(myfile, "r") as f:
    data = f.read()

with open(myfile, "w") as f:
    f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>", r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", data))

Neither truncate nor open(..., 'w') will change the inode number of the file (I tested twice, once with Ubuntu 12.04 NFS and once with ext4).

By the way, this is not really related to Python. The interpreter calls the corresponding low level API. The method truncate() works the same in the C programming language: See http://man7.org/linux/man-pages/man2/truncate.2.html

edited Sep 04 '19 at 03:10

Boris Verkhovskiy

14,854
11
100
103

answered Jul 13 '12 at 11:10

guettli

25,042
81
346
663

1

`Neither truncate nor open(..., 'w') will change the inode number of the file` why is it important? – rok Aug 03 '20 at 11:25
1

@rok if the inode changes or not is not relevant in most cases. Only in edge cases where you use hard-links, but [I advice to avoid hard links](https://github.com/guettli/programming-guidelines/blob/master/README.md#avoid-magic-or-uncommon-things). – guettli Aug 06 '20 at 12:21
4

is there a drawback of using the "f.seek() ..." approach over the "with open( ...)" approach? – Frankstar Mar 16 '21 at 15:17
@Frankstar I think it's even the other way around as you don't create two file objects but only one with the `f.seek()` approach, otherwise both behave the same, I think. – Lampe2020 Feb 19 '23 at 13:37

score 130 · Answer 2 · edited Sep 04 '19 at 02:01

130

file='path/test.xml' 
with open(file, 'w') as filetowrite:
    filetowrite.write('new content')

Open the file in 'w' mode, you will be able to replace its current text save the file with new contents.

edited Sep 04 '19 at 02:01

Moshe Rabaev

1,892
16
31

answered Dec 04 '18 at 07:38

Chikku Jacob

2,114
1
18
33

10

This is a good way to clear a file and write something new to it, but the question was about reading the file, modifying the contents and overwriting the original with the new contents. – Boris Verkhovskiy Sep 04 '19 at 02:39
2

@Boris, what is the problem with reading the file first and then using the code in this answer? – Rayhunter Nov 25 '20 at 10:58
1

@Rayhunter : it's inefficient – serv-inc Nov 27 '20 at 06:43
2

it's simple and efficient, does the job in a perfect way. – Chikku Jacob Dec 03 '20 at 04:01

score 23 · Answer 3 · edited Oct 22 '18 at 09:17

23

Using truncate(), the solution could be

import re
#open the xml file for reading:
with open('path/test.xml','r+') as f:
    #convert to string:
    data = f.read()
    f.seek(0)
    f.write(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>",data))
    f.truncate()

edited Oct 22 '18 at 09:17

Community

1
1

answered May 23 '17 at 14:40

serv-inc

35,772
9
166
188

4

`seek` *and* `truncate`!!! I couldn't figure out why `seek` alone was not working. – conner.xyz Mar 11 '20 at 21:44
@conner.xyz Maybe I am wrong but seek is responsible to change the **cursor** position. And `write` is responsible for writing into the file from the **cursor** position. **write** doesn't care about after writing a file is there any content remain or not. `truncate` here do the job to remove the rest of the content from the **cursor** position of the file. – Almabud Feb 01 '22 at 11:08
1

@Almabud, I just tested `with open(...) as f: f.truncate() f.write(...)` (no `seek(0)`) and it does indeed seem to replace the file contents. – conner.xyz Feb 02 '22 at 01:14
@conner.xyz recently I was trying your solution as it is simpler but it wasn't working as expected. ` file = pyexiv_img = pyexiv2.ImageData(file.read()) pyexiv_img.clear_exif() file.truncate() file.write(pyexiv_img.get_bytes()) ` This is not working. I need to add `file.seek(0)` after then it works fine. – Almabud Feb 14 '22 at 16:59

score 5 · Answer 4 · answered Jan 11 '19 at 09:33

import os#must import this library
if os.path.exists('TwitterDB.csv'):
        os.remove('TwitterDB.csv') #this deletes the file
else:
        print("The file does not exist")#add this to prevent errors

I had a similar problem, and instead of overwriting my existing file using the different 'modes', I just deleted the file before using it again, so that it would be as if I was appending to a new file on each run of my code.

score 3 · Answer 5 · answered Jan 06 '20 at 15:32

3

See from How to Replace String in File works in a simple way and is an answer that works with replace

fin = open("data.txt", "rt")
fout = open("out.txt", "wt")

for line in fin:
    fout.write(line.replace('pyton', 'python'))

fin.close()
fout.close()

answered Jan 06 '20 at 15:32

7beggars_nnnnm

697
3
12

score 1 · Answer 6 · answered Nov 22 '22 at 12:31

1

in my case the following code did the trick

with open("output.json", "w+") as outfile:  #using w+ mode to create file if it not exists. and overwrite the existing content
    json.dump(result_plot, outfile)

answered Nov 22 '22 at 12:31

Ch Usman

519
6
9

score 0 · Answer 7 · answered Aug 03 '20 at 13:08

Using python3 pathlib library:

import re
from pathlib import Path
import shutil

shutil.copy2("/tmp/test.xml", "/tmp/test.xml.bak") # create backup
filepath = Path("/tmp/test.xml")
content = filepath.read_text()
filepath.write_text(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", content))

Similar method using different approach to backups:

from pathlib import Path

filepath = Path("/tmp/test.xml")
filepath.rename(filepath.with_suffix('.bak')) # different approach to backups
content = filepath.read_text()
filepath.write_text(re.sub(r"<string>ABC</string>(\s+)<string>(.*)</string>",r"<xyz>ABC</xyz>\1<xyz>\2</xyz>", content))

Replace and overwrite instead of appending

7 Answers7

Linked

Related