1

I'm integrating rspec-puppet tests in our puppet project and I'm trying to auto generate "should compile" specs for all hosts (and eventually probably other resource). This is to ensure everything at least compiles successfully.

Given a list of nodes I can do this with:

hosts.each do |host|
  describe host, type: :host do
    it { should compile }
  end
end

The issue is how to actually get the hosts list. I can use regex to parse the node files but that obviously leads to insanity... Since the puppet gem is already present and used for loading the catalogues by the rspec-puppet gem, can I use it to get a list of the hosts?

EDIT:

I eventually managed to do this using the puppet pops system but I'm not sure if this the best way to do this or if there's some higher level abstraction that would be easier to use:

require 'spec_helper'
require 'puppet/pops'

code_dirs = [RSpec.configuration.module_path, RSpec.configuration.manifest_dir]
definitions =
    Dir["{#{code_dirs.join(',')}}/**/*.pp"].
        map {|file| Puppet::Pops::Parser::Parser.new.parse_file file}.
        flat_map {|parsed_manifest| parsed_manifest.definitions}

hosts = definitions.
    select {|definition| definition.is_a? Puppet::Pops::Model::NodeDefinition}.
    flat_map {|node_definition| node_definition.host_matches}.
    select {|host_match| host_match.is_a? Puppet::Pops::Model::LiteralString}.
    map {|string_host_match| string_host_match.value}

classes = definitions.
    select {|definition| definition.is_a? Puppet::Pops::Model::HostClassDefinition}.
    map {|host_class_definition| host_class_definition.name}

hosts.each do |host|
  describe host, type: :host do
    it {should compile}
  end
end

classes.each do |klass|
  describe klass, type: :class do
    it {should compile}
  end
end
eirc
  • 1,654
  • 14
  • 14
  • 1
    This is outside the realm of unit testing (rspec-puppet) and falls within the domain of acceptance testing. Look into Beaker and Test-Kitchen (recommended) with the providers Serverspec (recommended), Goss, and Testinfra. That will be much easier for you. It may also fall within the realm of smoke testing if you want less robustness than acceptance testing, which means check out octocatalog-diff. That would also be much easier for you. – Matthew Schuchard Sep 14 '17 at 13:22
  • Thanks for the recommendations, it's been long since I last got into puppet testing and most were not in my radar. I eventually came up with a scheme that seems ok: I created a `rake smoke:generate` task with pops like in the example I posted, to create simple `should compile` spec files for every class, host and define in a `spec/smoke` folder. This should at least help us stop committing code that doesn't even compile. I think octocatalog could help with, as they mention, seeing what's new in the catalog before running on real hosts. Integration tests with beaker & friends are coming soon. – eirc Sep 15 '17 at 08:55
  • 1
    Yeah I think octocatalog-diff is the closest to what you want, except that they made it with the idea of catalog compilation and not module. However, it can be tweaked a little for just a module, and you can easily iterate across nodes with it. Github's SRE team uses a few lines of Ruby to parallelize their tests across all nodes. Finally, Kevin Paulisse is doing a great job maintaining and developing it, so you know it will be supported. – Matthew Schuchard Sep 15 '17 at 13:17

1 Answers1

1

Have a look at https://github.com/nwops/puppet-retrospec , which provides parsing manifests and templating tests over existing modules.

David Schmitt
  • 58,259
  • 26
  • 121
  • 165