Capybara+Selenium+Docker confusion: Remote Firefox hangs

Question

We've been stumped trying to figure out how to get Selenium JavaScript testing working with our Dockerised Ruby (Hanami) app. For spec examples that don't depend on JavaScript, of course, the Capybara-default Rack::Test driver is adequate, but every modern app depends on JavaScript for one thing or another, and ours is/will be no different.

I've read a few of the answers here on SO, and started rifling through GitHub looking for random similar projects, but we've been stopped for an increasingly politically unsustainable length of time here.

To summarise:

1. We can run JavaScript-using feature specs successfully if no non-JS specs are run in the same rake invocation. (We use jbodah/minitest-tagz to filter spec invocations);
2. When running our full set of specs, Firefox Marionette hangs when running a JavaScript-enabled example, with its log message indicating that it does not have the IP address it should be connecting to.

We've only found a very limited set of tutorials/walkthroughs that are obviously applicable, mostly quite long in the tooth by 2018. The two that we've cargo-culted most blatantly are

1. Integration testing with dockerized Selenium and Capybara, by Ahmet Kizilay in February 2016; and
2. Dockerized Rails Capybara tests on top of Selenium, by Alfredo Motta in May, 2016.

It's probable, or at least likely, that we've consistently misread or glossed over something in these that is biting us now. It's also worth noting that Docker internals, particularly related to networking, seem to have evolved significantly in the last two years, and that may have something to do with our problem.

I've included copies of the following files in a Gist, since they're too much in aggregate to paste here:

1. 00-Capybara-Selenium-Docker-confusion__readme.md: this problem statement;
2. 01-docker-compose.yml: the docker-compose.yml file for the application;
3. 02-run_tests.sh: the scripts/run_tests.sh file which serves as the command run by the web container as specified in the docker-compose file;
4. 03-features_helper.rb: the `spec/features_helper' required by each feature spec; primarily concerned with Selenium setup;
5. 04-Dockerfile: the Dockerfile used to build the web container;
6. 05-join_as_member_spec.rb: one of the feature specs which exercises JavaScript in the application;
7. 06-focused_logfile.log: the terminal output from running docker-compose up --exit-code-from web with only JavaScript-exercising spec examples being run;
8. 07-unfocused_logfile.log: the terminal output from running the same docker-compose invocation with all examples in all specs enabled (i.e., not limited through the use of tag :focus).

Any helpful comments and/or pointers to existing working test setups (regardless of framework being used, e.g., Rails) would be greatly appreciated. Thanks.

Answer 1

Finally! We fixed the problem, in a highly unexpected way. Below is the text (excluding static-analysis report boilerplate) of the commit we've just pushed to our project repo. Thanks for the helpful comments from all, particularly /u/nyekks on Reddit.

Fix the Selenium prob that wasn't a Selenium prob

It's been a long road, getting from there to here...

We'd been chasing our tail for days, never mind how many, trying to figure out why our Selenium JavaScript tests were apparently breaking Docker. Eventually, we posted pleas for help on Reddit, StackOverFlow, and Gitter; all variations of (and referencing) this Gist.

Turns out the problem wasn't (wholly) with Selenium, Capybara, Docker, and/or the interaction between the three. @mitpaladin eventually discovered a randomisation seed (15474) that, when specified via a TESTOPTS environment variable passed to Rake in scripts/run_tests.sh, would yield a green bar. That was Interesting, but not necessarily directly Useful; by proving that there was at least a 1 in ((2**333) - 1) chance of a green bar, the tests themselves weren't irreparably breaking anything. (Your chances, however, are far better with your local lottery.) As we both engaged in further online research, @mitpaladin again came through, finding multiple StackOverflow answers like this one that pointed the (middle) finger directly at...DatabaseCleaner of all things. To be fair, their README lightly touches on the problem but, if you don't already know that's what you're looking for, you probably aren't going to think twice about it even if you read it.

Changing the DatabaseCleaner.strategy from :transaction to :deletion allowed everything to work; after making the change, we were immediately rewarded with a green bar. (There is a DB_CLEANER_STRATEGY constant defined in spec/spec_helper.rb, however, Just In Case we ever need to change it again.)

Now maybe we can get our heads back on and figure out what we were trying to accomplish after this newly-passing feature spec lo! these many days ago.