Read and write file atomically

Question

I'd like to read and write a file atomically in Ruby between multiple independent Ruby processes (not threads).

I found atomic_write from ActiveSupport. This writes to a temp file, then moves it over the original and sets all permissions. However, this does not prevent the file from being read while it is being written.
I have not found any atomic_read. (Are file reads already atomic?)

Do I need to implement my own separate 'lock' file that I check for before reads and writes? Or is there a better mechanism already present in the file system for flagging a file as 'busy' that I could check before any read/write?

^{The motivation is dumb, but included here because you're going to ask about it.}

^{I have a web application using Sinatra and served by Thin which (for its own reasons) uses a JSON file as a 'database'. Each request to the server reads the latest version of the file, makes any necessary changes, and writes out changes to the file.}

^{This would be fine if I only had a single instance of the server running. However, I was thinking about having multiple copies of Thin running behind an Apache reverse proxy. These are discrete Ruby processes, and thus running truly in parallel.}

^{Upon further reflection I realize that I really want to make the act of read-process-write atomic. At which point I realize that this basically forces me to process only one request at a time, and thus there's no reason to have multiple instances running. But the curiosity about atomic reads, and preventing reads during write, remains. Hence the question.}

Checking and then acting (as with lock file) is prone to race conditions. You don't want to separate the two. — Sergio Tulentsev, Apr 17 '15 at 16:42

score 3 · Accepted Answer · answered Apr 17 '15 at 16:54

3

You want to use File#flock in exclusive mode. Here's a little demo. Run this in two different terminal windows.

filename = 'test.txt'

File.open(filename, File::RDWR) do |file|
  file.flock(File::LOCK_EX)

  puts "content: #{file.read}"
  puts 'doing some heavy-lifting now'
  sleep(10)
end

answered Apr 17 '15 at 16:54

Sergio Tulentsev

226,338
43
373
367

Thanks, this looks great. When is the lock released? When the file handle is closed? Also, the documentation says "not available on all platforms". Do you have any idea what platforms it is/is not available on? – Phrogz Apr 17 '15 at 17:06
I suppose. But you can unlock explicitly by calling `file.flock(File::LOCK_UN)` – Sergio Tulentsev Apr 17 '15 at 17:08
As for the platform support: I *think* it's something that unixes would have. Features tend not to work on windows, but I did a similar thing on windows ~10 years ago. Not in ruby, though, but the OS had the capabilities back then. You should try :) – Sergio Tulentsev Apr 17 '15 at 17:10
Watch out for filesystems too, especially shared filesystems like nfs – Frederick Cheung Apr 17 '15 at 20:55
I'm going to accept this for now, as it looks like the right answer. The ambiguities around OS and file system limitations make me think that a better answer might exist. If you have one, post it, and perhaps I'll switch the acceptance. – Phrogz Apr 20 '15 at 20:45
Looking into possibly using this today, and I hear it's bad practice to utilize locks in *nix systems, at least exclusive ones, in case the process dies and you can no longer possibly access the file, until I'm guessing, you restart the system. Instead, I should be making a temporary file and renaming it to achieve an atomic operation. – Pysis Nov 15 '17 at 16:39

Nakilon · Answer 2 · 2018-10-01T07:12:30.707

1

Take a look at transaction and open_and_lock_file methods in "pstore.rb" (Ruby stdlib).

YAML::Store works fine for me. So when I need to read/write atomically I (ab)use it to store data as a Hash.

edited Oct 01 '18 at 07:12

answered Aug 26 '18 at 15:42

Nakilon

34,866
14
107
142

Read and write file atomically

2 Answers2

Linked