How to isolate Puppet function mocking in rspec unit tests

Question

I have a Puppet class that uses the result of a custom Puppet function. To make sure I only test the logic in my class, and not the logic in my function when doing unit tests for the class, I want to mock the function.

However, I can't seem to fully isolate my mocked function to a single context. My real testing code is bigger than the following example, but I've boiled it down to this:

class break_tests {

  $result = my_mocked_function('foo', 'bar', 'baz')

  file { 'under_test':
    content => $result,
  }

}

require 'spec_helper'

def mock_mmf(return_value)
  Puppet::Parser::Functions.newfunction(:'my_mocked_function', type: :rvalue) do |_args|
    return return_value
  end
end

# rubocop:disable Metrics/BlockLength
describe 'break_tests' do
  context 'numero uno' do
    before { mock_mmf('foo') }
    it { should contain_file('under_test').with_content('foo') }
  end
  context 'numero duo' do
    before { mock_mmf('bar') }
    it { should contain_file('under_test').with_content('bar') }
  end
end

Failures:

  1) break_tests numero duo should contain File[under_test] with content  supplied string
     Failure/Error: it { should contain_file('under_test').with_content('bar') }
       expected that the catalogue would contain File[under_test] with content set to supplied string
     # ./spec/classes/break_tests_spec.rb:17:in `block (3 levels) in <top (required)>'

I tried splitting it up into two describes and even two separate files, the result is always the same: one context receives the output from a different context.

In my bigger test case, with about 20 tests, it's even more complex, seemingly influenced by whether or not some contexts have facts assigned to them. Ordering of the contexts does not seem to matter.

What am I missing here?

I suspect the reason for this is that you are not really mocking/stubbing the Puppet function, but rather creating a custom invocation of it in your test. The return would persist in your second `describe` block I would think, and that would explain what you are seeing. Unless you are modifying the catalog with your function, you can safely ignore it in these tests. — Matthew Schuchard, Jul 02 '19 at 14:50
I would be inclined to guess that you indeed are not obtaining the isolation you want, on account of the same catalog builder instance being used for all tests. Likely only the first-evaluated attempt to define your function actually succeeds. That could explain the results from your example; it's unclear to me whether it would explain your bigger test case. — John Bollinger, Jul 02 '19 at 21:35

Alex Harvey · Accepted Answer · 2019-07-03T23:39:07.967

At the time of writing (Puppet 6.6.0, Rspec-puppet 2.7.5), the whole business of mocking Puppet functions remains all a bit of a mess unfortunately. It doesn't help that rspec-puppet's docs still refer to the legacy Ruby API for functions.

The problem that you're facing is as John Bollinger has said in the comments, that you have a compiler instance that runs when the Rspec files are loaded, and then assertions in it blocks that run later.

Remember that Rspec (Rspec itself, nothing to do with Puppet) runs in two phases:

The describe and context blocks are all evaluated at the time the Rspec files are loaded.
The it blocks, the examples themselves, are cached and evaluated later.

There is an answer on this by Rspec's author at Stack Overflow here that I recommend having a look at.

So, to avoid the catalog being compiled for every single example - which would make Rspec-puppet way too slow - the compilation is cached prior to the it examples being executed.

So what can you do?

Option 1 - Use Tom Poulton's rspec-puppet-utils.

This has the advantage of a ready-made solution that takes care of mocking your Puppet functions through a well known interface, and using the expected feature Tom has implemented, you can also cause the catalogs to be recompiled in different examples.

The disadvantages could be that it uses Mocha rather than Rspec-mocks, it uses the legacy Ruby API - but then so do Rspec-puppet's docs! - and it hasn't been committed to since 2017.

Thus you could rewrite your tests this way:

require 'spec_helper'
require 'rspec-puppet-utils'

def mock_mmf(return_value)
  MockFunction.new('my_mocked_function').expected.returns(return_value)
end

describe 'test' do
  context 'numero uno' do
    before { mock_mmf('foo') }
    it { should contain_file('under_test').with_content('foo') }
  end
  context 'numero duo' do
    before { mock_mmf('bar') }
    it { should contain_file('under_test').with_content('bar') }
  end
end

Option 2 - Steal some of Tom's code - monkey patch Rspec-puppet

Under the hood however, Tom's code just monkey patches Rspec-puppet, and you could just steal the little bit that does that and refactor your examples like this:

require 'spec_helper'
require 'rspec-puppet/cache'

module RSpec::Puppet  ## Add this block
  module Support
    def self.clear_cache
      @@cache = RSpec::Puppet::Cache.new
    end
  end
end

def mock_mmf(return_value)
  RSpec::Puppet::Support.clear_cache  ## ... and this line
  Puppet::Parser::Functions.newfunction(:'my_mocked_function', type: :rvalue) do |_args|
    return return_value
  end
end

describe 'test' do
  context 'numero uno' do
    before { mock_mmf('foo') }
    it { should contain_file('under_test').with_content('foo') }
  end
  context 'numero duo' do
    before { mock_mmf('bar') }
    it { should contain_file('under_test').with_content('bar') }
  end
end

Option 3 - find a better way

If you search around in other Puppet modules for long enough, you may find better solution - even solutions that use the Puppet 4 Function API. That said, I guess it doesn't matter so much for the purpose of your test, as long as the fake function returns the response you expect.

Thank you so much for this clear answer. It's weird though, I have another module where I used the exact same mocking technique to mock puppetdb_query results to great success. The above solution works there as well, so I'm going to switch to it as clearly my way of doing things is not reliable, even if it did work in one location. — Simon, Jul 03 '19 at 12:11
Did your other code also require the stubbed function to return different values in different examples? @Simon — Alex Harvey, Jul 03 '19 at 12:19
Yes, wildly different values. It uses the output of a query for resources to decide whether or not to apply more resources to other nodes, to roll out features gradually. It filters over tags and counts results, so my test code has different outputs for the query with varying resource contents and amounts — Simon, Jul 04 '19 at 13:10

How to isolate Puppet function mocking in rspec unit tests

1 Answers1

Option 1 - Use Tom Poulton's rspec-puppet-utils.

Option 2 - Steal some of Tom's code - monkey patch Rspec-puppet

Option 3 - find a better way