4

I'm trying to write a script that stores some data chunks inside flat .txt files (they're small files, less than 100 lines).

Anyway, I'm trying to, in effect, update a single matching line with a new value for that line while leaving everything else alone in the file but cannot quite figure out how to modify just 1 line rather than replacing the full file.

Here is my code so far:

# get file contents as array.
array_of_lines = File.open( "textfile.txt", "r" ).readlines.map( &:chomp )

line_start = "123456:"    # unique identifier
new_string = "somestring" # a new string to be put after the line_start indentifier.

# cycle through array finding the one to be updated/replaced with a new line.
# the line we're looking for is in format 123456:some old value

# delete the line matching the line_start key
array_of_lines.delete_if( |line| line_start =~ line )

# write new string into the array.
array_of_lines.push( "#{line_start}:#{new_string}" )

# write array contents back to file, replacing all previous content in the process
File.open( "textfile.txt", "w" ) do |f|
    array_of_lines.each do |line|
        f.puts line
    end
end

The textfile.txt contents will always be consisting of the format:

unique_id:string_of_text

where I can match the unique_id using app data generated by the script to figure out which line of text to update.

Is there a better way of doing what I'm trying to?

It seems a little inefficient to read the entire file into memory, looping over everything just to update a single line in that file.

Andrew Marshall
  • 95,083
  • 20
  • 220
  • 214
Jannis
  • 17,025
  • 18
  • 62
  • 75
  • 1
    See also: http://stackoverflow.com/questions/4397412/read-edit-and-write-a-text-file-line-wise-using-ruby/4399299#4399299 – Wayne Conrad Jul 14 '12 at 23:44
  • Thanks Wayne, Good to know about those potential performance/memory issues. – Jannis Jul 14 '12 at 23:59
  • Don't use `readlines` unless you KNOW the file will fit into memory. If it's larger than the available memory for the script, then processing will go to a crawl. Instead use `foreach` or `each_line` to read the file line-by-line. It's faster and scalable. http://stackoverflow.com/a/25189286/128421 – the Tin Man Nov 26 '14 at 18:39

1 Answers1

4

You can't do what you want unless the new data you are writing is the same length as the old data.

If the length is different then all the bytes in the file after your modification need to be moved. Moving file data always involves rewriting everything (from the point of the modification onwards). In that case you might as well rewrite the whole file since your files are so small.

If the replacement data is the same length, then you can use IO.seek to put the file pointer to the appropriate location, and then just use write to enter the replacement data.

If you still don't want to rewrite the whole file, but instead just move the data around (if replacement length is different), then you need to seek to the correct location and then write everything to the end of the file from that point forward. If the replacement is shorter you will also need to call File.truncate to resize the file.

Casper
  • 33,403
  • 4
  • 84
  • 79
  • Thanks for the answer. Since I'm so new to Ruby it's good to know that I am on the right track here. One question in regards to rewriting the file: Is it more efficient to concat the `array_of_lines` into a single string first via `array_of_lines.join("\n")` and then write only this to the file once, or is the way I've done it, writing each line inside an each loop to the file, just as good/efficient? – Jannis Jul 14 '12 at 23:37
  • @Jannis - It's hard to say which one is more efficient. I'd write them separately like you do. Joining just creates one more temporary string in memory. However the **best option** is just to realize you can call `puts` with the whole array: `f.puts array_of_lines`. No need to loop or join at all. – Casper Jul 14 '12 at 23:44
  • Also you might want to look at this for a one-liner that also does what you want: http://stackoverflow.com/questions/1274605/ruby-search-file-text-for-a-pattern-and-replace-it-with-a-given-value – Casper Jul 14 '12 at 23:45
  • awesome thanks for the tip! `f.puts array_of_lines` it is! Having had a look at the `gsub` one-liner linked, I presume that doing it that way would not honor line breaks necessarily? – Jannis Jul 14 '12 at 23:58
  • @Jannis - it runs the `gsub` once per each line in the file. So it works pretty much as expected ("replace this with that for each line in the file"). The comments there explain the command-line options and what essentially happens. – Casper Jul 15 '12 at 00:04
  • Ah nice, just gave it a try now and works great. Thanks again. – Jannis Jul 15 '12 at 00:17