How to replace a column in a txt file?

Question

I have a txt file,

k1=np.linspace(0,1,10)
k2=np.linspace(0,2,10)
k3=np.linspace(0,3,10)
k4=np.linspace(0,4,10)

np.savetxt('k.txt',np.transpose([k1,k2,k3,k4]))

Now I wanna replace the second column (k2) to a new array

k5=np.linspace(0,5,10)

How to efficiently replace the column without reusing np.savetxt or a loop?

I'm not sure I understand the question. Are you asking about the numpy arrays, or how to use Python to edit the script itself? — Barmar, Jun 20 '18 at 21:09
So you want the second column to be replaced with `k5` in your file? — user3483203, Jun 20 '18 at 21:11
@kinderchan It's still unclear what you're asking about. The question says you want to modify the file, not the array. — Barmar, Jun 20 '18 at 22:59
@Barmar Yes, I wanna modify the file with replacing a specific column. — kinder chen, Jun 20 '18 at 23:05
Text files don't have columns. They have lines, and lines are made up of characters. — Barmar, Jun 20 '18 at 23:09
If you consider each character to be a column, you're replacing two columns: the `2` in `k2` and the `2` in `(0,2,10)` — Barmar, Jun 20 '18 at 23:11

score 0 · Answer 1 · answered Jun 20 '18 at 21:17

It's not easy to do without rewriting the entire file. Text files are simple, sequential method of data storage, without fixed structure, indexing, or random access search. So you'd have to implement all those features.

The common approach is to read the file, edit it, and overwrite the file with the data the way you want.

Another approach when the file is too big to fit entirely in memory is to create a new temporary file, and read the old file line by line in a loop, while you write to the new file. Then delete the old file and rename the new file to have the same name as the original.

If you need more than that, I suggest using a database. sqlite is included in python and can do what you request directly inside a file, really fast.

This is a simple example, my original files are much more complicated. At first I considered the way to reopen , reedit and then rewrite the file as well, but I just wonder if there is a easy way to just replace a specific column. — kinder chen, Jun 20 '18 at 23:03
@kinderchan as I said in the answer, **no**, you can't easily replace/delete part of a sequential file, because it is sequential. — nosklo, Jun 21 '18 at 15:08

Maxwell77 · Answer 2 · 2018-06-23T11:27:19.590

def change(tuple):
    line, newk = tuple
    oldk = line.split()
    return "%s %.18e %s %s\n" % (oldk[0], newk, oldk[2], oldk[3])

filename = "k.txt" 
lines = fileinput.input(filename)
tuples = zip(lines, k5)
newlines = list(map(change, tuples))

fo = open(filename, "r+")
fo.writelines(newlines)

Here the script creates a list of tuples(line, k) containing for each line the new value for k.

%.18e is the default format for the NumPy.savetext() method, so you need to adapt the format to same one used for creating your source file.

score -2 · Answer 3 · answered Jun 20 '18 at 23:18

You don't need a loop. You can read the whole file into a list with readlines(), modify a specific element of the list, then write it back out to the file with writelines().

with open("filename.txt", "r+") as f:
    lines = f.readlines()
    lines[2] = "k5=np.linspace(0,5,10)\n"
    f.seek(0)
    f.writelines(lines)
    f.truncate()

f.seek() goes back to the beginning of the file, so that f.writelines() overwrites it. Then we use f.truncate() to remove any excess content if the replacement is shorter than the original file contents.

Don't forget the \n in the new contents of lines[2]. writelines() doesn't put newlines between the strings in its argument.

How to replace a column in a txt file?

3 Answers3