Testing multiple hosts with the same test using serverspec

Question

The Advanced Tips section of the Serverspec site shows an example of testing multiple hosts with the same test set. I've built an example of my own (https://gist.github.com/neilhwatson/81249ad393800a76a8ad), but there are problems.

The first problem is that the tests stop at the first failure rather than proceeding through the lot and keeping a tally. The second is that the failure output does not indicate on which host the failure occurred. What can I do to fix these problems and produce a final report for all hosts?

score 2 · Answer 1 · edited May 23 '17 at 11:58

2

For the first issue, ServerSpec by default will run all your tests. However, since you have a loop that executes a Rake task for each environment, the first environment to have a failure causes the task to fails and so an exception is raised and the rest of your tasks don't run.

I've forked your gist and updated the Rake task to surround it with a begin/rescue.

...
begin
  desc "Run serverspec to #{host}"
  RSpec::Core::RakeTask.new(host) do |t|
    ENV['TARGET_HOST'] = host
    t.pattern = "spec/base,cfengine3/*_spec.rb"
  end
rescue
end
...

For the second problem, it doesn't look like ServerSpec will output which environment the tests are running in. But since the updated Gist shows that the host gets set in the spec_helper.rb we can use that to add an RSpec configuration that sets up an after(:each) and only output the host on errors. The relevant code changes are in a fork of the gist, but basically you'll just need the below snippet in your spec_helper.rb:

RSpec.configure do |c|
  c.after(:each) do |example|
    if example.exception
       puts "Failed on #{host_run_on}"
    end
  end
end

edited May 23 '17 at 11:58

Community

1
1

answered May 25 '15 at 02:57

Arthur Maltson

5,760
4
30
33

Closer, but I still can't tell what host failure happens on. New gist: https://gist.github.com/neilhwatson/1d41c696102c01bbb87a – Neil H Watson May 25 '15 at 13:04
A new error: `example` is not available from within an example (e.g. an `it` block) or from constructs that run in the scope of an example (e.g. `before`, `let`, etc). It is only available on an example group (e.g. a `describe` or `context` block). – Neil H Watson May 27 '15 at 12:12
Ahh, my bad, that's from RSpec 2, not RSpec 3. It's done this way in RSpec 3: http://stackoverflow.com/a/26164289/238292. I'll update the response. – Arthur Maltson May 28 '15 at 02:39
Nope: https://gist.github.com/neilhwatson/81249ad393800a76a8ad#file-gistfile1-rb-L88-L106 – Neil H Watson May 28 '15 at 12:28
OK, that tells me `host` is `nil`. Can you try the way I created a `host_run_on` temporary variable instead? https://gist.github.com/amaltson/e870a3634db587de947d#file-serverspec-rb-L50-L61 – Arthur Maltson May 28 '15 at 13:35
Still returns nil hostname. – Neil H Watson May 28 '15 at 15:50
So I don't get it... how does ServerSpec get that host name. If the value is nil, how does ServerSpec know which host to log into? – Arthur Maltson May 29 '15 at 03:00

Testing multiple hosts with the same test using serverspec

1 Answers1