2

I'm writing a developer tool for Ruby and as part of testing it against various gems, I'm running bundle commands like bundle cache and bundle install --local, which modify the state inside the Ruby installation (AFAICT this modification this is not preventable -- but I don't fully understand all the flags to the various bundle subcommands).

Specifically the sequence of commands for testing a single gem is as follows:

# First get gem-under-test's dependencies
# This step is mostly an optimization, so
# that any modified directories can be cached
# and this step can be skipped when only
# my-tool.gem is modified.
cd gem-under-test && bundle cache

cp /path/to/my-tool.gem vendor/cache/

# Tweak Gemfile+Gemfile.lock to mark my-tool as a dependency
git apply /path/to/gem-under-test.patch

# Mimic an actual user running 'bundle install' after
# updating their Gemfile with my-tool as a dependency
bundle install --local

# Check that my-tool.gem works properly, mimicking
# what a user of my-tool would do
bundle exec my-tool

I'd like the tests to run in parallel, so the shared mutation of the Ruby installation is undesirable. The naive thing to do would be to have multiple Ruby installations which are isolated, and test using those. However, having multiple Ruby installations seems to be complicated by the fact that Ruby installations are not location-independent. From the ruby/setup-ruby README:

The default tool cache directory (/opt/hostedtoolcache on Linux, /Users/runner/hostedtoolcache on macOS, C:/hostedtoolcache/windows on Windows) must be writable by the runner user. This is necessary since the Ruby builds embed the install path when built and cannot be moved around. (emphasis added)

This means that having multiple Ruby installations at different locations in the filesystem requires building Ruby from source for each test (instead of building it once and copying the installation around), which is too slow to be practical.

What is a good way to solve the problem of running bundle invocations in parallel on the same base toolchain but with isolation from one another?

I took a look at few projects which seem potentially related:

  • There is rbenv-gemset but that seems to key off a separate .rbenv-gemsets file and there is no mention of Bundler in the README. I would like to not convert from Gemfile/Gemfile.lock/.gemspec files if possible (for simplicity)
  • There is Homebrew's portable-ruby. From the sparse info in the README, it's unclear to me if it can be used like a standard Ruby installation. If this can fill the "location-independent" bit, then I can copy the installation directory once per test.

Another option is to go for a more heavy-weight solution like building a Docker image first with Ruby and its dependencies, and running tests inside an image. However, that adds a bunch of complexity in terms of setup + worse debuggability (compared to simply having separate directories).

typesanitizer
  • 2,505
  • 1
  • 20
  • 44
  • 1
    I'm not sure I fully understand the workflow, but can't you install _all the gems to test against_ at once and parallelize only the tests themselves? – Konstantin Strukov Aug 15 '22 at 12:43
  • Added more details about the workflow. Wrt "install all the gems" -- I think that _might_ work. It wouldn't do full isolation; I'd be relying on Bundler to enforce that. But that seems like a reasonable assumption. I could still potentially break up the "install all gems" into multiple stages to better cache intermediate results (to avoid invalidating it when only `my-tool.gem` changes). – typesanitizer Aug 16 '22 at 01:16

0 Answers0