MinitTest Testcase fails when I run complete suite, succeeds when I run it alone

Question

I am deeply confused about my tests and one testcase specifically:

When I run all of my integration tests together this specific testcase gives me this error:

    UsersSignupCapybaraTest
    test_signup_process_with_capybara                              ERROR (5.16s)
    Capybara::ElementNotFound: Unable to find link or button "Sign up now!"

When running just this one test it passes:

    UsersSignupCapybaraTest
    test_signup_process_with_capybara                               PASS (10.19s)

Can someone explain this to me? I asked a similar question yesterday here. I think I am not understanding some basic mechanism of my tests. Am I wrong, assuming that each testcase is tested isolated? Or does one start, where there previous stopped? That wouldn't make sense as I would have to take care of the order they get executed, which sounds not right to me.

Here is the file containing the testcase:

    require 'test_helper'
    class UsersSignupCapybaraTest  :chrome)
        end
        Capybara.current_driver = :selenium_chrome
      end
      test "signup process with capybara" do
        visit root_path
        click_on "Sign up now!"
        fill_in "user_name",    with: "Neuer User"
        fill_in "user_email",   with: "neuer@user.de"
        # more code ...
      end
    end
    
    

Here is my test_helper.rb

    # set to test environment
    ENV['RAILS_ENV'] ||= 'test'
    # load up the rails application
    require File.expand_path('../../config/environment', __FILE__)
    # start minitest
    require 'rails/test_help'
    require 'minitest/rails'
    require 'minitest/rails/capybara'
    require 'capybara/rails'
    require 'capybara/poltergeist'
    require 'minitest/reporters'

    Minitest::Reporters.use!(
      Minitest::Reporters::SpecReporter.new,
      ENV
    )

    class ActiveSupport::TestCase
      ActiveRecord::Migration.check_pending!
      # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
      fixtures :all

      # Returns true if a test user is logged in.
      def is_logged_in?
        !session[:user_id].nil?
      end

      # Logs in a test user.
      def log_in_as(user, options = {})
        password    = options[:password]    || 'password'
        remember_me = options[:remember_me] || '1'
        if integration_test?
          post login_path, session: { email:       user.email,
                                      password:    password,
                                      remember_me: remember_me }
        else
          session[:user_id] = user.id
        end
      end

      private

      # Returns true inside an integration test.
      def integration_test?
        defined?(post_via_redirect)
      end
    end

    class ActionDispatch::IntegrationTest
      # Make the Capybara DSL available in all integration tests
      include Capybara::DSL

      # Logs in a test user.
      def log_in_as(user, options = {})
        password    = options[:password]    || 'password'
        remember_me = options[:remember_me] || '1'
        if integration_test?
          post login_path, session: { email:       user.email,
                                      password:    password,
                                      remember_me: remember_me }
        else
          session[:user_id] = user.id
        end
      end
    end

    # https://github.com/jnicklas/capybara#transactions-and-database-setup
    class ActiveRecord::Base  
      mattr_accessor :shared_connection
      @@shared_connection = nil

      def self.connection
        @@shared_connection || retrieve_connection
      end
    end  
    ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection 


    # register Capybara driver: Selenium
    Capybara.register_driver :selenium_chrome do |app|
      Capybara::Selenium::Driver.new(app, :browser => :chrome)
    end
    Capybara.current_driver = :selenium_chrome
    Capybara.default_wait_time = 5 

Answer 1

In general this happens when some tests create some state in your database or another data layer. In your case I suspect it to have something to do with the user session ? Maybe some other test sign in some user that is not properly signed out. If you use devise/warden maybe this is of some use for you.

A good thing to start with is to maybe try add save_and_open_page to your tests (docs) after the visit root_path and look for the state of the page when the test suite runs and check if the "Sign up now!"-Button is there.

Answer 2

you need a

def teardown
  Capybara.reset_sessions!
  Capybara.use_default_driver
end

either in the test class or one of its parents - Its probably best to define a separate class

class CapybaraTest < ActionDispatch::IntegrationTest
  include Capybara::DSL

  def teardown
    Capybara.reset_sessions!
    Capybara.use_default_driver
    super  # this might not be necessary - or should maybe be before the other two methods - not sure off the top of my head
  end
end

and then derive tests that use Capybara from CapybaraTest