Installing Ruby

You can use the Ruby version that comes with your operating system. If you are using the Vagrant testing setup documented in this book, it is easy to install Ruby into the system packages:

[vagrant@client ~]$ sudo yum install -y ruby-devel rubygems rake libxml2-devel

Guides for installing Ruby can be found in Appendix C.

Install the bundler gem for local installation of necessary dependencies:

$ sudo gem install bundler --no-ri --no-rdoc
Fetching: bundler-1.11.1.gem (100%)
Successfully installed bundler-1.11.1
1 gem installed

Adding Beaker

Add the following lines to the Gemfile in the module directory:

gem 'beaker-rspec', :require => false
gem 'pry',          :require => false

These lines will ensure that Beaker is installed along with the other dependencies in the next step. Beaker gem dependencies require these development libraries to compile binary extensions:

[vagrant@client ~]$ sudo yum install -y gcc-d++ libxml2-devel libxslt-devel

Bundling Dependencies

If you haven’t done this already, you’ll need to install the puppetlabs_spec_helper and other dependency gems. The best way to do this is to run the bundler install command within the module directory. bundle will read the Gemfile and pull in rspec, rspec-puppet, beaker, and all of their dependencies. These are testing and template creation tools that simplify test creation:

[vagrant@client puppet]$ bundler install
Fetching gem metadata from https://rubygems.org/..........
Fetching version metadata from https://rubygems.org/..
Resolving dependencies...
Installing rake 11.1.1
Installing CFPropertyList 2.2.8
Using diff-lcs 1.2.5
...snip a long list of gems...
Installing rspec 3.4.0
Installing puppet 4.10.9
Installing rspec-puppet 2.3.2
Installing puppetlabs_spec_helper 1.1.1
Installing beaker 2.37.0
Installing beaker-rspec 5.3.0
Bundle complete! 6 Gemfile dependencies, 95 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.

Defining Fixtures

Create a .fixtures.yml file to define the testing fixtures (dependencies) and where to acquire them for testing purposes. The information in this file should duplicate the dependencies in metadata.json.

The top of the file should always be the same. This tells the testing frame to copy the current module from its directory:

fixtures:
  symlinks:
    puppet: "#{source_dir}"

Then define each dependency for your module and the minimum version you support. You can list their names on the Puppet Forge or their GitHub URL. The following two examples will have similar effects. From the Forge:

  forge_modules:
    stdlib:
      repo: "puppetlabs/stdlib"
      ref: 4.5.1

From GitHub:

  repositories:
    stdlib:
      repo: "git://github.com/puppetlabs/puppetlabs-stdlib"
      ref: "4.5.1"

If you are testing development of multiple modules, you may want to use symlinks to the source tree for each. Assuming the dependency is in the same directory structure:

  symlinks:
    some_dependency: "#{source_dir}/../some_dependency"

Test that dependency setup worked properly like so:

$ rake spec
(in /etc/puppetlabs/code/environments/test/modules/puppet)
Notice: Preparing to install into
  /etc/puppetlabs/code/environments/test/modules/puppet/spec/fixtures/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/code/environments/test/modules/puppet/spec/fixtures/modules
└── puppetlabs-stdlib (v3.2.1)
/usr/bin/ruby -I/usr/lib/ruby/gems/1.8/gems/rspec-support-3.2.2/lib:...
No examples found.

Finished in 0.00027 seconds (files took 0.04311 seconds to load)
0 examples, 0 failures

This shows that all fixtures (dependencies) were installed, but no examples (tests) were available. Let’s start building one now.

Defining the Main Class

Within your module directory, change into the spec/classes/ directory. Inside this directory, create a file named <modulename>_spec.rb.

[vagrant@client puppet]$ cd spec/classes
[vagrant@client classes]$ $EDITOR puppet_spec.rb

First, we will define a test where the module test builds (compiles) a catalog successfully with the default options:

require 'spec_helper'

describe 'puppet', :type => 'class' do

  context 'with defaults for all parameters' do
    it { is_expected.to contain_class('puppet') }
    it { is_expected.to contain_class('puppet::params') }
    it { is_expected.to compile.with_all_deps }
  end
end

Let’s go ahead and run the testing suite against this very basic test:

[vagrant@client puppet]$ rake spec
(in /etc/puppetlabs/code/modules/puppet)
ruby -I/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/gems/rspec-support-3.2.2/lib:
..

Finished in 10.22 seconds (files took 0.56629 seconds to load)
2 examples, 0 failures

Now that our basic test passed, let’s go on to start checking the input parameters.

Passing Valid Parameters

What if we expand the tests to include every possible input value? Rather than repeating each test with a different value, we use Ruby loops to iteratively build each of the tests from an array of values:

['1','0'].each do |repo_enabled|
  ['emerg','crit','alert','err','warning','notice','info'].each do |loglevel|
    context "with repo_enabled = #{repo_enabled}, loglevel #{loglevel}" do
      let :params do
        {
          :repo_enabled => repo_enabled,
          :loglevel     => loglevel,
        }
      end

      it { is_expected.to
        contain_package('puppet-agent').with({'version' => '1.10.9-1'})
      }
    end
  end
end

Whoa, look at that. We added 15 lines of code and yet it’s performing 36 more tests:

[vagrant@client puppet]$ rake spec
(in /etc/puppetlabs/code/modules/puppet)
ruby -I/opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/gems/rspec-support-3.2.2/lib:

......................................

Finished in 10.53 seconds (files took 0.56829 seconds to load)
38 examples, 0 failures

Failing Invalid Parameters

Always test to ensure incorrect values fail. This example shows two tests that are intended to fail:

  context 'with invalid loglevel' do
    let :params do
      {
        :loglevel => 'annoying'
      }
    end

    it { is_expected.to compile.with_all_deps }
  end

  context 'with invalid repo_enabled' do
    let :params do
      {
        :repo_enabled => 'EPEL'
      }
    end

    it { is_expected.to compile.with_all_deps }
  end

Run the tests to see what error messages are kicked back:

[vagrant@client puppet]$ rake spec
Failures:

1) puppet4 with invalid loglevel should build a catalog w/o dependency cycles
   Failure/Error: should compile.with_all_deps
     error during compilation: Parameter loglevel failed on Class[Puppet]:
       Invalid value "annoying". Valid values are debug, info, notice, warning,
       err, alert, emerg, crit, verbose.  at line 1
   # ./spec/classes/puppet_spec.rb:52:in 'block (3 levels) in <top (required)>'

2) puppet4 with invalid repo_enabled should build a catalog w/o dependency cycles
   Failure/Error: should compile.with_all_deps
     error during compilation: Expected parameter 'repo_enabled' of
       'Class[Puppet]' to have type Enum['0', '1'], got String at line 1
   # ./spec/classes/puppet_spec.rb:65:in 'block (3 levels) in <top (required)>'

Finished in 10.81 seconds (files took 0.57989 seconds to load)
40 examples, 2 failures

Now let’s change the expect lines to accept the error we expect:

    it do
      is_expected.to compile.with_all_deps.and_raise_error(Puppet::Error,
        /Invalid value "annoying". Valid values are/
      )
    end

and the following for the test of repo_enabled:

    it do
      is_expected.to compile.with_all_deps.and_raise_error(Puppet::Error,
        /Expected parameter 'repo_enabled' .* to have type Enum/)
      )
    end

Now when you run the tests, you will see the tests were successful because the invalid input produced the expected error.

Testing File Creation

You can test to ensure that a file resource is created. The simplest form is:

  it { is_expected.to contain_file('/etc/puppetlabs/puppet/puppet.conf') }

Now, this only checks that the file exists, and not that it was modified correctly by the module. Test resource attributes using this longer form:

  it do
    is_expected.to contain_file('/etc/puppetlabs/puppet/puppet.conf').with({
      'ensure' => 'present',
      'owner'  => 'root',
      'group'  => 'root',
      'mode'   => '0444',
    })
  end

Finally, you can also check the content against a regular expression. Here’s an example where we pass in a parameter, and then want to ensure it would be written to the file:

  let :params do
  {
    :loglevel => 'notice',
  }
  end

  it do
    is_expected.to contain_file('/etc/puppetlabs/puppet/puppet.conf').with_content({
      /^\s*loglevel\s*=\s*notice/
    })
  end

Validating Class Inclusion

You can test to ensure that a dependent class was loaded:

  it { is_expected.to contain_class('puppet::_config') }

When testing a defined type, set a title for the defined type to be passed during the test:

  let(:title) { 'mytype_testing' }

Using Facts in Tests

Some manifests or tests may require that certain facts are defined properly. Inside the context block, define a hash containing the fact values you want to have available in the test:

let :facts do
  {
    :osfamily => 'RedHat',
    :os       => {
      'family' => 'RedHat',
      'release' => { 'major' => '7', 'minor' => '2' }
    },
  }
end

By default, the hostname, domain, and fqdn facts are set from the fully qualified domain name of the host. To adjust the node name and these three facts for testing purposes, add this to the test:

    let(:node) { 'webserver01.example.com' }

Using Hiera Input

Within your module directory, change to the spec/fixtures/ directory. Inside this directory, create a subdirectory named hiera, containing a valid hiera.yaml file for testing:

[vagrant@client puppet]$ cd spec/fixtures
[vagrant@client puppet]$ mkdir hiera
[vagrant@client classes]$ $EDITOR hiera/hiera.yaml

You can change anything you want that is valid for Hiera in this configuration file, except for the datadir, which should reside within the fixtures path. Unless you desire a specific change, the following file could be used unchanged in every module:

# spec/fixtures/hiera/hiera.yaml
---
:backends:
  - yaml
:yaml:
  :datadir: /etc/puppetlabs/code/hieradata
:hierarchy:
  - os/"%{facts.os.family}"
  - common

Now, add the following lines to a test context within one of the class spec files:

  let(:hiera_config) { 'spec/fixtures/hiera/hiera.yaml' }
  hiera = Hiera.new( :config => 'spec/fixtures/hiera/hiera.yaml' )

Now create your Hiera input files. The only necessary file is spec/fixtures/hiera/common.yaml. The others can be added only when you want to test things:

---
puppet::loglevel      : 'notice'
puppet::repo_enabled  : '1'
puppet::agent::status : 'running'
puppet::agent::enabled: true

You can use this Hiera data to configure the tests:

  let :params do
    {
      :repo_enabled => hiera.lookup('puppet::repo_enabled',nil,nil),
      :loglevel     => hiera.lookup('puppet::loglevel',nil,nil),
    }
  end

This configuration allows you to easily test the common method of using Hiera to supply input parameters for your modules.

Defining Parent Class Parameters

In some situations, your module will depend upon a class that requires some parameters to be provided. You cannot set parameters or use Hiera for that class, because it is out of scope for the current class and test file.

The workaround is to use a pre_condition block to call the parent class in resource-style format. Pass the necessary parameters for testing as parameters for the resource declaration, and this module instance will be created before your module is tested.

Here is an example from my mcollective module, which had to solve this exact problem:

describe 'mcollective::client' do
  let(:pre_condition) do
    'class { "mcollective":
      hosts           => ["middleware.example.net"],
      client_password => "fakeTestingClientPassword",
      server_password => "fakeTestingServerPassword",
      psk_key         => "fakeTestingPreSharedKey",
    }'
  end

...tests for the mcollective::client class...

Testing Functions

Unit tests should be created for any functions added by the module. Each function test should exist in a separate file, stored in the spec/functions/ directory of the module, and named for the function followed by the _spec.rb extension.

At a bare minimum, the test should ensure that:

• The function fails for insufficient or too many values.
• Given an expected input, it produces an expected output.

Here is an example that should work for our make_boolean() example:

#! /usr/bin/env ruby -S rspec
require 'spec_helper'

describe 'make_boolean' do
  describe 'should raise a ParseError if there is less than 1 argument' do
    it { is_expected.to run.with_params().and_raise_error(Puppet::ParseError) }
  end

  describe "should convert string '0' to false" do
    it { is_expected.to run.with_params("0").and_return(false) }
  end
end

Adding an Agent Class

Within the spec/classes/ directory, create a file named agent_spec.rb. This is an exercise for you. Build the agent class, testing every valid and invalid input just like we did for the Puppet class.

For this, we simply want to test that the package, config file, and service resources are all defined:

require 'spec_helper'

describe 'puppet::agent', :type => 'class' do

  context 'with defaults for all parameters' do
    it do
      is_expected.to contain_package('puppet-agent').with({ 'version' => 'latest' })
      is_expected.to contain_file('puppet.conf').with({ 'ensure' => 'file' })
      is_expected.to contain_service('puppet').with({ 'ensure' => 'running' })
    end

    it { is_expected.to compile.with_all_deps }
  end
end

We have demonstrated how to build tests. Now, you should build out more tests for valid and invalid input.

Testing Other Types

Every object type provided by a module can be tested. Place tests for the other types in the directories specified in Table 17-1.

Installing Ruby for System Tests

As Beaker will need to run vagrant commands to create virtual machines to run the test suites on, you’ll need to run Beaker on your development system rather than one of the virtual machines you’ve been using so far. Following are the steps to set up and appropriate testing environment on your system.

1. Follow the instructions for installing Ruby on your desktop operating system. Directions for installing Ruby on different operating systems can be found in Appendix C.
2. Use the gem install bundler command to install Bundler.
3. Change into the module directory and run bundle install to install the test dependencies.

Defining the Nodeset

Create a directory within the module to contain the nodesets used for testing:

[vagrant@client puppet]$ mkdir -p spec/acceptance/nodesets

Create a file name default.yml within this directory. This file should contain YAML data for nodes to be created for the test. The following example will create a test node using the same Vagrant box utilized within this book.

HOSTS:
  centos-7-x64:
    roles:
	  - agent
    platform: el-7-x86_64
    box: puppetlabs/centos-7.2-64-nocm
    hypervisor: vagrant
    vagrant_memsize: 1024

Create additional nodeset files within this directory for all platforms that should be tested. Sample nodeset files for different operating systems can be found at Example Vagrant Hosts Files.

Configuring the Test Environment

Create a file named spec/spec_helper_acceptance.rb within the module directory. This file defines the tasks needed to prepare the test system.

The following example will ensure Puppet and the module dependencies are installed:

require 'beaker-rspec'
require 'pry'

step "Install Puppet on each host"
install_puppet_agent_on( hosts, { :puppet_collection => 'pc1' } )

RSpec.configure do |c|
  # Find the module directory
  module_root = File.expand_path( File.join( File.dirname(__FILE__), '..') )

  # Enable test descriptions
  c.formatter = :documentation

  # Configure all nodes in nodeset
  c.before :suite do
    # Install module and dependencies
    puppet_module_install(
      :source      => module_root,
      :module_name => 'puppet',
    )
    hosts.each do |host|
      # Install dependency modules
      on host, puppet('module', 'install', 'puppetlabs-stdlib'),
         { :acceptable_exit_codes => [0,1] }
      on host, puppet('module', 'install', 'puppetlabs-inifile'),
         { :acceptable_exit_codes => [0,1] }
    end
  end
end

This is generally a consistent formula you can use with any module. The adjustments specific to this module have been bolded in the preceding example.

Creating an Acceptance Test

The tests are written in rspec, like the unit tests created in the previous section. However ServerSpec tests are also available and can be used in combination with rspec tests.

For most modules that set up services, the first test should be an installation test to validate that it installs with the default options, and that the service is properly configured.

Create a file named spec/acceptance/installation_spec.rb within the module directory. The following example defines some basic tests to ensure that the package is installed and that the service runs without returning an error:

require 'spec_helper_acceptance'

describe 'puppet class' do
  context 'default parameters' do
    # Using puppet_apply as a helper
    it 'should install with no errors using default values' do
      puppetagent = <​<-EOS
        class { 'puppet::agent': }
      EOS

      # Run twice to test idempotency
      expect( apply_manifest( puppetagent ).exit_code ).to_not eq(1)
      expect( apply_manifest( puppetagent ).exit_code ).to eq(0)
    end

    describe package('puppet-agent') do
      it { should be_installed }
    end

    describe file('/etc/puppetlabs/puppet/puppet.conf') do
      it { should be_a_file }
    end

    describe service('puppet') do
      it { should be_enabled }
    end

  end
end

You can and should create more extensive tests that utilize different input scenarios for the module. Each test should be defined as a separate file in the spec/acceptance/ directory, with a filename ending in _spec.rb.

You can find more information, including details about writing good tests, in these places:

• The best documentation for Beaker can be found at puppetlabs/beaker on GitHub.
• The best documentation for writing rspec tests can be found at rspec-core on the rspec.info site.
• The best documentation for writing serverspec tests can be found at serverspec resource types on the serverspec.org site.

Running Acceptance Tests

To spin up virtual machines to run the test, you’ll need to run the acceptance tests from a system that has Vagrant installed. Your personal system will work perfectly fine for this purpose.

$ bundle exec rspec spec/acceptance
Hypervisor for centos-7-x64 is vagrant
Beaker::Hypervisor, found some vagrant boxes to create
==> centos-7-x64: Forcing shutdown of VM...
==> centos-7-x64: Destroying VM and associated drives...
created Vagrantfile for VagrantHost centos-7-x64
Bringing machine 'centos-7-x64' up with 'virtualbox' provider...
==> centos-7-x64: Importing base box 'puppetlabs/centos-7.2-64-nocm'...
Progress: 100%
==> centos-7-x64: Matching MAC address for NAT networking...

puppet::agent class
  default parameters

    should install with no errors using default values
    Package "puppet-agent"
      should be installed
    File "/etc/puppetlabs/puppet/puppet.conf"
      should be a file
    Service "puppet"
      should be enabled

Destroying vagrant boxes
==> centos-7-x64: Forcing shutdown of VM...
==> centos-7-x64: Destroying VM and associated drives...

Finished in 17.62 seconds (files took 1 minute 2.88 seconds to load)
4 examples, 0 failures

  2) puppet::agent class default parameters
       Package "puppet-agent" should be installed
     Failure/Error: it { should be_installed }
       expected Package "puppet-agent" to be installed

     # ./spec/acceptance/installation_spec.rb:17:in `block (4 levels) in <top>'

$ BEAKER_destroy=no BEAKER_debug=yes bundle exec rspec spec/acceptance

$ BEAKER_set=centos-6-x86 BEAKER_debug=yes rspec spec/acceptance

* Install Puppet on each host

centos-7-x64 01:36:36$ rpm --replacepkgs -ivh
  Retrieving http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
  Preparing...                          ########################################
  Updating / installing...
  puppetlabs-release-7-11               ########################################
  warning: rpm-tmp.KNaXPi: Header V4 RSA/SHA1 Signature, key ID 4bd6ec30: NOKEY

centos-7-x64 executed in 0.35 seconds

    describe package('puppet-agent') do
      it { should be_installed }
    end
    # Access host console to debug failure
    binding.pry

$ BEAKER_destroy=no bundle exec rspec spec/acceptance
...
From: learning-puppet4/puppet4/spec/acceptance/installation_spec.rb @ line 20 :

    16:     describe package('puppet-agent') do
    17:       it { should be_installed }
    18:     end
    19:     # Make the debug console available
 => 20:     binding.pry
    21:

[1] pry(RSpec::ExampleGroups::PuppetAgentClass::DefaultParameters)>

>> cat spec/acceptance/installation_spec.rb
>> edit spec/acceptance/installation_spec.rb

>> shell-mode
learning-puppet4/puppet4 $ .ls spec/acceptance/
installation_spec.rb   nodesets/

>> hosts.each_with_index do |host,i| print "#{i} #{host.hostname()}\n"; end;
1 centos-7-x64

>> on hosts[0], 'rpm -qa |grep puppet' ;

centos-7-x64 02:50:14$ rpm -qa |grep puppet
  puppet-3.8.4-1.el7.noarch
  puppetlabs-release-7-11.noarch