Find all current files that Ruby opened when running a program

Question

This is someone of an esoteric request, and is just of theoretical interest to me (In other words, I am not used this in production code). I'd like to know if it is possible to figure out all the files involved when running a ruby program. That is, files required by said program during its execution (or involved some other way, such as autoloaded by Rails midway through the program).

I know that one can use ObjectSpace and loop through the object, excluding all but File and its descendants, and I had partial success with this method when I disable garbage collection (otherwise the list shrinks at random times, understandably). However, doing this at the start of the program ignores those files that are required in the middle of it, so I can run the same code at the end, and get the union of said files?

One thing to try is patching out the `require` function with one that logs, but keep in mind there are a number of ways to load in Ruby code: `load`, `autoload`, `require_relative` among others. You might have more success with wrapping Ruby in a debugger and watching for file I/O events. — tadman, Feb 01 '17 at 20:33
to find all dependencies of a gem (but not all files) see http://stackoverflow.com/questions/21108109/how-do-i-find-out-all-the-dependencies-of-a-gem — peter, Feb 01 '17 at 22:23

score 3 · Answer 1 · edited Apr 13 '17 at 12:36

lsof

On linux, you could use lsof :

puts `lsof -a -p #{Process.pid}`

It gives you the files held open by the current process.

The list is far from being complete, though : Once this line executes, many files have been read, parsed, executed and closed.

When lsof scans for open files, they don't appear in the list anymore.

strace

strace (also on Linux) is a great tool for what you want to achieve (see this thread). It will give a lot (possibly way too much) information :

strace -e trace=open -o opened_files.txt ruby hello_world.rb
#=> Hello world

opened_files.txt now begins with :

open("/home/ricou/.rvm/rubies/ruby-2.3.1/lib/tls/x86_64/libruby.so.2.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/ricou/.rvm/rubies/ruby-2.3.1/lib/tls/libruby.so.2.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/ricou/.rvm/rubies/ruby-2.3.1/lib/x86_64/libruby.so.2.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/ricou/.rvm/rubies/ruby-2.3.1/lib/libruby.so.2.3", O_RDONLY|O_CLOEXEC) = 3
open("/home/ricou/.rvm/rubies/ruby-2.3.1/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/home/ricou/.rvm/rubies/ruby-2.3.1/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
open("/home/ricou/.rvm/rubies/ruby-2.3.1/lib/libgmp.so.10", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
....

It has 571 lines for a "Hello World" script and 112302 (!) for a "rails runner hello_world.rb" script.

The output file has many "ENOENT (No such file or directory)" lines, so you might want to parse and filter it.

For macOS and other *nix, there should be dtrace.

A lot of these files are opened only briefly, so catching them in `lsof` may prove futile. It's good for seeing what's held open, though. — tadman, Feb 01 '17 at 21:46
`strace` and `dtrace` are really great tools. Much better call! I've also had luck with [`fs_usage`](https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/fs_usage.1.html) for macOS/OS X. — tadman, Feb 01 '17 at 21:48

KARASZI István · Answer 2 · 2017-02-01T21:16:42.737

I did exactly the same a couple months ago. Here is my dirty little program:

REQUIRE_FILE = ENV.fetch('REQUIRE_FILE', '/tmp/requires.txt')

File.open(REQUIRE_FILE, 'w')

Kernel.module_eval do
  alias_method :require_without_benchmark, :require

  def require(name)
    start = Time.now
    begin
      require_without_benchmark(name)
    ensure
      time = Time.now - start
      File.write(REQUIRE_FILE, [name, time.to_f].join(',').concat("\n"), mode: 'a')
    end
  end
end

Then run your program as:

ruby -r ./require-benchmark.rb program.rb

score 1 · Answer 3 · edited May 23 '17 at 12:02

See How do I get a list of files that have been `required` in Ruby? for required files. For all the dependencies of a gem see How do I find out all the dependencies of a gem?

The constant $LOADED_FEATURES holds an array of all files with paths that are loaded with a require, not just the gems themselves but all files involved. Try $LOADED_FEATURES.dup.uniq at the end of your script.

For the normal files opened by File you could overwrite IO.open and/or File.open and sister methods read, binread etc in the way KARASZI shows in his answer.

Find all current files that Ruby opened when running a program

3 Answers3

lsof

strace