Handling Change

The reason that you need to cast aside imperative programming is to handle change better.

When you write code that performs a sequence of operations, that sequence will make the desired change the first time it is run. If you run the same code the second time in a row, the same operations will either fail or create a different state than desired. Here’s an example:

$ sudo useradd -u 1001 -g 1001 -c "Joe User" -m joe
$ sudo useradd -u 1001 -g 1000 -c "Joe User" -m joe
useradd: user 'joe' already exists

So then you need to change the code to handle that situation:

# bash excerpt
getent passwd $USERNAME > /dev/null 2> /dev/null
if [ $? -ne 0 ]; then
    useradd -u $UID -g $GID -c "$COMMENT" -s $SHELL -m $USERNAME
else
    usermod -u $UID -g $GID -c "$COMMENT" -s $SHELL -m $USERNAME
fi

OK, that’s six lines of code and all we’ve done is ensure that the username isn’t already in use. What if we need to check to ensure the UID is unique, the GID is valid, and that the password expiration is set? You can see that this will be a very long script even before we adjust it to ensure it works properly on multiple operating systems.

This is why we say that imperative programming doesn’t handle change very well. It takes a lot of code to cover every situation you need to test.

Using Idempotence

When managing computer systems, you want the operations applied to be idempotent, where the operation achieves the same results every time it executes. Idempotence allows you to apply and reapply (or converge) a configuration manifest and always achieve the desired state.

In order for imperative code to be idempotent, it needs to have instructions for how to compare, evaluate, and apply not just every resource, but also each attribute of the resource. As you saw in the previous section, even the simplest of operations will quickly become ponderous and difficult to maintain.

Declaring Final State

As we mentioned in Introduction, for a configuration state to be achieved no matter the conditions, the configuration language must avoid describing the actions required to reach the desired state. Instead, the configuration language should describe the desired state itself, and leave the actions up to the interpreter. Language that declares the final state is called declarative.

Rather than writing extensive imperative code to handle every situation, it is much simpler to declare what you want the final state to be. In other words, instead of including dozens of lines of comparison, the code reflects only the desired final state of the resource (a user account, in this example). Here we will introduce you to your first bit of Puppet configuration language, a resource declaration for the same user we created earlier:

user { 'joe':
  ensure     => present,
  uid        => '1001',
  gid        => '1000',
  comment    => 'Joe User',
  managehome => true,
}

As you can see, the code is not much more than a simple text explanation of the desired state. A user named Joe User should be present, a home directory for the user should be created, and so on. It is very clear, very easy to read. Exactly how the user should be created is not within the code, nor are instructions for handling different operating systems.

Declarative language is much easier to read, and less prone to breakage due to environment differences. Puppet was designed to achieve consistent and repeatable results. You describe what the final state of the resource should be, and Puppet will evaluate the resource and apply any necessary changes to reach that state.

Reviewing Declarative Programming

Conventional programming languages create change by listing exact operations that should be performed. Code that defines each state change and the order of changes is known as imperative programming.

Good Puppet manifests are written with declarative programming. Instead of defining exactly how to make changes, in which you must write code to test and compare the system state before making that change, you instead declare how it should be. It is up to the Puppet agent to evaluate the current state and apply the necessary changes.

As this chapter has demonstrated, declarative programming is easier to create, easier to read, and easier to maintain.

Installing Vagrant

If you are going to follow the recommendation to use Vagrant, let’s get started installing it. You’ll need to download two packages.

Go to VirtualBox Downloads and download the appropriate platform package for your system.

Next, go to Vagrant Downloads and download the appropriate platform package for your system.

Install these packages according to the instructions for your operating system. I’ve included detailed instructions for Windows and Mac users in the following subsections. If you’re running Vagrant on Linux or another platform, I’m going to assume you are expert enough to handle this installation yourself.

Installing Vagrant on Mac

First, you should run the VirtualBox installer. Open the VirtualBox DMG image file you downloaded and click on the Virtualbox.pkg installer, as shown in Figure 2-1.

Accept the license and the installer will complete the installation.

Next, you should run the Vagrant installer. Open the Vagrant DMG image file you downloaded and click on the Vagrant.pkg installer, as shown in Figure 2-2.

Figure 2-1. VirtualBox package installer for Mac

Figure 2-2. Vagrant package installer for Mac

Likewise, accept the license agreement and the installer will complete the installation.

Finally, if you don’t already have it, you will need to install Xcode on your machine. Perform the following steps:

1. Pull down on the Apple logo at the top left of your screen.
2. Select “App Store...”
3. In the App Store, type xcode into the search bar on the right.
4. The top-left application should be Xcode Developer Tools.
5. Click GET and then click INSTALL APP beneath its name.

Installing Git Tools on Windows

On Windows platforms, you must install Git and its tools in order to use Vagrant successfully.

Open your browser to Git Source Code Management and click the Windows Build on the lower right of the page. Run the installer when the download completes.

Accept the default components.

On the “Adjusting your PATH environment” screen, select “Use Git from the Windows Command Prompt,” as shown in Figure 2-3.

Figure 2-3. Select “Use Git from the Windows Prompt”

On the “Configuring the line ending conversions” screen (as shown in Figure 2-4), any option will work successfully for this book. Select the default option if you are unsure. You can change this later.

Figure 2-4. Choose which line endings you prefer

Select the MinTTY terminal emulator on the following screen (Figure 2-5).

Figure 2-5. Choose the MinTTY terminal for the Bash shell

Accept the final prompts to complete the installation.

Installing VirtualBox on Windows

Vagrant will use the VirtualBox virtualization engine to run the test systems. Installing VirtualBox is very straightforward. Run the VirtualBox installer package that you downloaded on the previous page.

Click Next to start the installation, and then click Next again to install all features (Figure 2-6).

Figure 2-6. Install all features (the default choice)

You can disable the options to create shortcuts (Figure 2-7), as these won’t be necessary for the learning environment.

Figure 2-7. Select only the “Register file associations” checkbox

Accept the warning about a short interruption to networking (Figure 2-8). In most situations, you won’t notice any effect.

Figure 2-8. VirtualBox network interruption notice

To install VirtualBox, click Install, as shown in Figure 2-9.

Figure 2-9. Install VirtualBox

At this point, you’ll be prompted to install several virtualization drivers (Figure 2-10). You can either choose to always trust software from Oracle, or click Install for each one.

Figure 2-10. Oracle drivers

Disable the checkbox to start VirtualBox after installation (Figure 2-11). Vagrant will start it automatically.

Figure 2-11. Disable autostart

Installing Vagrant on Windows

Now we should install Vagrant. Run the Vagrant installer package that you downloaded (Figure 2-12).

Figure 2-12. Vagrant installer for Windows

You’ll need to accept the license agreement, as shown in Figure 2-13.

Figure 2-13. License agreement

Select where you want to install Vagrant. As Figure 2-14 shows, I prefer to install Vagrant in the normal system location for 64-bit programs. It doesn’t matter which path you choose here.

Figure 2-14. Select an installation path for Vagrant

Once you get to the confirmation screen acknowledging completion of the installation, you can exit the Setup Wizard by clicking Finish (Figure 2-15).

Figure 2-15. Click Finish to complete the installation

Windows systems will need to reboot, as shown in Figure 2-16.

Figure 2-16. Restart after installation

Starting a Bash Shell

At this point, you’ll need to start a Bash prompt. We’ll be using Bash for the remainder of this book, so now is a good time to become comfortable using it.

On a Macintosh, follow these steps:

1. Open Finder.
2. In the sidebar on the left, click on Applications.
3. Open the Utilities folder.
4. Start the Terminal application.

On Windows 7 and before, follow these steps:

1. From the taskbar, click on the Windows Start button.
2. Select All Programs.
3. Select Git.
4. Click on Git Bash.

On Windows 8, follow these steps:

1. From the Metro desktop, click the down-arrow icon at the bottom left.
2. Scroll to the right and locate the Git section.
3. Click on Git Bash.

On Windows 10 (see Figure 2-17), follow these steps:

1. From the taskbar, click the Windows Start button.
2. Select the first item, All Apps.
3. Scroll to G.
4. Expand the arrow next to Git.
5. Click on Git Bash.

Figure 2-17. Windows 10: Git Bash

On Linux or any other platform, use the terminal prompt of your choice.

No matter which operating system you are using, the instructions assume you will start at a command prompt in your home directory.

Downloading a Box

To save some time later, you can download the virtual box image used as the base system for our learning environment (as mentioned before, we’ll use CentOS 7, as it is well supported by all Puppet Labs programs and modules):

$ vagrant box add --provider virtualbox centos/7
==> box: Loading metadata for box 'centos/7'
    box: URL: https://vagrantcloud.com/centos/7
==> box: Adding box 'centos/7' (v1707.01) for provider: virtualbox
    box: Downloading: https://vagrantcloud.com/centos/boxes/7/versions/1707.01
==> box: Successfully added box 'centos/7' (v1707.01) for 'virtualbox'!

You may note that Vagrant Cloud contains many boxes that already have Puppet installed. For this book you will go through the installation of Puppet on a stock system so that you have that experience.

Cloning the Learning Repository

Now we’ll download a repository that contains virtual machine configurations and learning materials. Starting in your home directory, use git to clone the following repository:

$ git clone https://github.com/jorhett/learning-puppet4
Cloning into 'learning-puppet4'...
remote: Counting objects: 238, done.
remote: Total 238 (delta 0), reused 0 (delta 0), pack-reused 238
Receiving objects: 100% (238/238), 38.50 KiB | 0 bytes/s, done.
Resolving deltas: 100% (110/110), done.

$ cd learning-puppet4/

As shown, change into the learning-puppet4 directory. All further instructions will assume you are in this directory.

Install the Vagrant vbguest Plugin

Now you should install the vagrant-vbguest plugin. This plugin ensures that the VirtualBox extensions on the virtual machine are kept up to date.

$ vagrant plugin install vagrant-vbguest
Installing the 'vagrant-vbguest' plugin. This can take a few minutes...
Fetching: vagrant-share-1.1.9.gem (100%)
Fetching: micromachine-2.0.0.gem (100%)
Fetching: vagrant-vbguest-0.14.2.gem (100%)

In particular, this plugin helps avoid problems that happen when a new kernel is installed, such as the /vagrant shared mount not being available.

Initializing the Vagrant Setup

The learning repository contains a Vagrantfile that lists the systems we’ll use in this book. If you are familiar with Vagrant, you’ll know that we can easily start these systems with vagrant up. We’ll do that now to initialize a client system for learning Puppet:

$ vagrant up client
Bringing machine 'client' up with 'virtualbox' provider...
==> client: Importing base box 'centos/7'...
==> client: Matching MAC address for NAT networking...
==> client: Checking if box 'centos/7' is up to date...
==> client: Setting the name of the VM: learning-puppet4_client_1504376676610_6140
==> client: Clearing any previously set network interfaces...
==> client: Preparing network interfaces based on configuration...
    client: Adapter 1: nat
    client: Adapter 2: hostonly
==> client: Forwarding ports...
    client: 22 (guest) => 2222 (host) (adapter 1)
==> client: Running 'pre-boot' VM customizations...
==> client: Booting VM...
==> client: Waiting for machine to boot. This may take a few minutes...
    client: SSH address: 127.0.0.1:2222
    client: SSH username: vagrant
    client: SSH auth method: private key
    client:
    client: Vagrant insecure key detected. Vagrant will automatically replace
    client: this with a newly generated keypair for better security.
    client:
    client: Inserting generated public key within guest...
    client: Removing insecure key from the guest if it's present...
    client: Key inserted! Disconnecting and reconnecting using new SSH key...
==> client: Machine booted and ready!

On a first boot the vbguest plugin will install the Virtualbox Guest Additions. This requires installing some CentOS packages we’ll skip over, but it should end like this:

Copy iso file VBoxGuestAdditions.iso into the box /tmp/VBoxGuestAdditions.iso
mount: /dev/loop0 is write-protected, mounting read-only
Installing Virtualbox Guest Additions 5.1.26 - guest version is unknown
Verifying archive integrity... All good.
Uncompressing VirtualBox 5.1.26 Guest Additions for Linux...........
VirtualBox Guest Additions installer
Copying additional installer modules ...
Installing additional modules ...
vboxadd.sh: Starting the VirtualBox Guest Additions.

Could not find the X.Org or XFree86 Window System, skipping.
Redirecting to /bin/systemctl start  vboxadd.service
Redirecting to /bin/systemctl start  vboxadd-service.service
==> client: Checking for guest additions in VM...
==> client: Setting hostname...
==> client: Configuring and enabling network interfaces...
    client: SSH address: 127.0.0.1:2222
    client: SSH username: vagrant
    client: SSH auth method: private key
==> client: Rsyncing folder: /Users/jorhett/learning-puppet4/ => /vagrant
==> client: Running provisioner: shell...
    client: Running: inline script

We’ve started only a single machine to learn from, named client. There are several other machines available that we will utilize in future chapters:

$ vagrant status
Current machine states:

client                    running (virtualbox)
puppetmaster              not created (virtualbox)
puppetserver              not created (virtualbox)
dashboard                 not created (virtualbox)
web1                      not created (virtualbox)

This environment represents multiple VMs. The VMs are all listed
above with their current state. For more information about a specific
VM, run `vagrant status NAME`.

You can suspend, resume, and destroy these instances quite easily:

$ vagrant suspend client
==> client: Saving VM state and suspending execution...

$ vagrant resume client
==> client: Resuming suspended VM...
==> client: Booting VM...
==> client: Waiting for machine to boot. This may take a few minutes...
    client: SSH address: 127.0.0.1:2222
    client: SSH username: vagrant
    client: SSH auth method: private key
    client: Warning: Connection refused. Retrying...
==> client: Machine booted and ready!

$ vagrant destroy client
    client: Are you sure you want to destroy the 'client' VM? [y/N] n
==> client: The VM 'client' will not be destroyed, since the confirmation
==> client: was declined.

If you do destroy the instance, just run vagrant up client to create a new instance and start over. We’re not going to spend any more time covering Vagrant here, but remember suspend and resume when you want to stop your test environment but don’t want to have to restart from scratch.

Now that this is running, let’s log in to the client system and get started.

$ vagrant ssh client
[vagrant@client ~]$ 

If you are using Windows and a version of Vagrant below 1.7.5, it has a bug that prevents it from recognizing MinTTY properly. Upgrade to a more recent version.

If you are using Windows and not using Git Bash, you may get an error about SSH not being found. This is due to not having the Git binaries in your path. You can correct that for this command prompt like so:

C:\> set PATH=%PATH%;C:\Program Files (x86)\Git\bin

Verifying the /vagrant Filesystem

The folder learning-puppet4 from your personal system should be synced to the client node as /vagrant. Let’s do a quick verification that it shows up here. You should see directory contents similar to this:

$ ls /vagrant
bin  doc  etc-puppet  LICENSE  manifests README.md  systemd-puppet  Vagrantfile

If you don’t see that, read back in your terminal to see if there was an error message about rsync of the /vagrant directory.

Unfortunately this isn’t a book about VirtualBox, so I feel obliged to send you to the VirtualBox support forum for a solution.

Initializing Non-Vagrant Systems

If you are using a virtual machine of your own choice, we’ll need to take a couple of steps so that you can follow the instructions in this book. If possible, mount the learning-puppet4/ directory to the virtual machine as /vagrant/. If you’re not sure how to do this, you can simulate it with the following commands:

$ git clone https://github.com/jorhett/learning-puppet4
…
$ sudo ln -s /home/username/learning-puppet4 /vagrant

Throughout this book, you will see the following prompt shown in the examples:

[vagrant@client ~]$ 

You’ll need to mentally replace this with whatever your virtual node’s shell prompt is. If you wish to ensure it looks the same, you can use the following environment variable to sync them up:

$ export PS1='[vagrant@client \W]\$ ' 

Installing Some Helpful Utilities

I recommend installing the following helpful utilities on the virtual machine, as we’ll be using them in subsequent chapters:

[vagrant@client ~]$ sudo yum install git

Choosing a Text Editor

Before we install Puppet on your virtual system, stop and take a moment to ensure you have a good text editor available to use. There are two ways to use text editors:

Inside the virtual system

If you are comfortable using Unix text editors such as Vim, Emacs, or Nano, you can run your editor inside the virtual system.

On your desktop

The folder learning-puppet4 is mounted inside the virtual system as /vagrant. This means you can use a text editor of your choice to edit files inside this directory, and they will be visible and available to Puppet on your virtual system.

One issue for both Mac and Windows users is the handling of carriage returns and line feeds within files. Puppet and Git both work best when lines are terminated by line feeds without carriage returns, which is known as “Unix file format.” It is important to select an editor that can read and write files in this format without any surprises.

Many editors on Windows open Unix format files correctly, but will quietly replace the endings with a carriage return when saving. As your target nodes may expect line-feed–terminated files, this can cause file corruption on the target host. Likewise, the opposite problem can exist: if you are editing files to be written out to Windows nodes, then you may need to preserve the carriage returns in the file. If you have a mixed environment of Unix/Linux and Windows nodes, it is essential that your editor can handle both file formats.

The following subsections provide a list of editors we recommend. I have limited this list to only include editors that handle both file formats well. If you don’t have any Windows nodes, then you can use any editor you like.

[vagrant@client ~]$ echo "EDITOR=nano" >> ~/.bashrc

[vagrant@client ~]$ $EDITOR filename

[vagrant@client ~]$ nano filename

Reviewing the Learning Environment

In this chapter, you created a virtualized environment suitable for learning and testing Puppet without affecting any production or personal systems.

The default learning environment we recommended had you install the Git development tools, Vagrant, and VirtualBox to provide a virtualized CentOS system on which you can test and develop.

If you already have a virtualization platform that you prefer to work with, you are welcome to use that instead. We recommended using an operating system compatible with RedHat Enterprise Linux 6 or 7, as it is the best tested and guaranteed version for compatibility with Puppet Supported modules that we will be using throughout the book. After you feel confident in the use of Puppet, there are detailed notes for use on other platforms in Appendix A.

During the course of this chapter, you cloned a Git repository that contains Vagrant system definitions, some example manifests, and other helpful files we’ll use throughout this book.

We also discussed the need for editing text files, and how you can use either an editor on the virtualized system or an editor on your desktop to create and edit Puppet manifests.

This environment is yours to keep. It will be useful as you develop and test Puppet manifests and modules during the learning process.

Adding the Package Repository

Now we’ll install and enable a Puppet Labs Puppet Collection repository on your fresh new system. Our first step is to check what the latest Puppet Collection is. You can find this at “Using Puppet Collections” in the Puppet Labs online documents.

Get the URL for the Enterprise Linux 7 Collection and install that package as described in the documentation.

[vagrant@client ~]$ sudo yum install -y \
 http://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm

This installs and enables a Puppet Collection package repository, which contains the Puppet 4 package. After it has finished installing, you can confirm it is enabled with the following command:

[vagrant@client ~]$ sudo yum repolist
Loaded plugins: fastestmirror
...snip repository checks...
repo id                      repo name                                   status
base/7/x86_64                CentOS-7 - Base                             9,363
extras/7/x86_64              CentOS-7 - Extras                             451
puppetlabs-pc1/x86_64        Puppet Labs PC1 Repository el 7 - x86_64      128
updates/7/x86_64             CentOS-7 - Updates                          2,146
repolist: 12,088

This shows you that there were 128 packages in the puppetlabs-pc1 repository (when this book was last updated).

Installing the Puppet Agent

Now let’s go ahead and install the Puppet agent:

[vagrant@client ~]$ sudo yum install -y puppet-agent
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.sonn.com
 * extras: mirrors.loosefoot.com
 * updates: mirrors.sonic.net
Resolving Dependencies
--> Running transaction check
---> Package puppet-agent.x86_64 0:1.10.9-1.el7 will be installed
--> Finished Dependency Resolution

...snip lots of output...

Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : puppet-agent-1.10.9-1.el7.x86_64             1/1
  Verifying  : puppet-agent-1.10.9-1.el7.x86_64             1/1

Installed:
  puppet-agent.x86_64 0:1.10.9-1.el7

Complete!

Guides for installing Puppet on other platforms can be found in Appendix A.

Reviewing Dependencies

If you have installed previous versions of Puppet, you’re accustomed to seeing dependency packages installed along with Puppet. Puppet 4 uses an All In One (AIO) installer, where all dependencies are installed together with Puppet. You can view these in the new installation directory:

[vagrant@client ~]$ ls -la /opt/puppetlabs/bin/
total 0
drwxr-xr-x 2 root root 68 Apr 6 4:41 .
drwxr-xr-x 4 root root 29 Apr 6 4:41 ..
lrwxrwxrwx 1 root root 33 Apr 6 4:41 facter -> /opt/puppetlabs/puppet/bin/facter
lrwxrwxrwx 1 root root 32 Apr 6 4:41 hiera -> /opt/puppetlabs/puppet/bin/hiera
lrwxrwxrwx 1 root root 30 Apr 6 4:41 mco -> /opt/puppetlabs/puppet/bin/mco
lrwxrwxrwx 1 root root 33 Apr 6 4:41 puppet -> /opt/puppetlabs/puppet/bin/puppet

Unlike previous versions, the Puppet commands are not installed in /usr/bin, and won’t be available in your default path. This puppet-agent package adds the file /etc/profile.d/puppet-agent.sh, which will add /opt/puppetlabs/bin to your path. This file is automatically included by the Bash shell during initialization. Let’s use that now:

[vagrant@client ~]$ which puppet
/usr/bin/which: no puppet in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin)
[vagrant@client ~]$ source /etc/profile.d/puppet-agent.sh
[vagrant@client ~]$ which puppet
/opt/puppetlabs/bin/puppet

Let’s review the other programs you see in this directory besides Puppet:

Facter

Facter is a program that evaluates a system and provides a number of facts about it. These facts include node-specific information (e.g., architecture, hostname, and IP address), in addition to custom information from plugins provided by Puppet modules. For a sneak preview, run the command facter right now and look at all the information it has.

We’ll be covering how to make use of Facter facts in Chapter 5, and how to create custom facts in Part II.

Hiera

Hiera is a component we’ll use to load in the data used by Puppet manifests and modules. Hiera allows you to provide default values and then override or expand them through a customizable hierarchy. This may sound complex, but Hiera’s beauty and elegance comes from its simplicity.

You’ll learn how to use Hiera in Part II, and after its introduction, you will see Hiera used in every subsequent example in the book.

Marionette Collective

Marionette Collective, or MCollective, is an orchestration framework tightly integrated with Puppet.

You’ll learn how to use it in Part IV, where we’ll use the mco client to manipulate the Puppet agent.

Reviewing Puppet 4 Changes

If you have installed previous versions of Puppet, you’ll notice that the installation packages and the configuration file paths have all changed.

Making Tests Convenient

Throughout this book, you’ll edit files and make changes within the Puppet configuration directory. Normally, this would require you to type sudo every time you want to change a file.

I have found that most people prefer the following change to create an easier-to-use environment for learning (this command will make the vagrant user the owner of all files in that directory):

[vagrant@client ~]$ sudo chown -R vagrant /etc/puppetlabs

The remainder of this book assumes you have made this change. If you decide to continue using sudo, you’ll need to add it to any command involving files in /etc/puppetlabs.

[vagrant@client ~]$ sudo chown -R vagrant /etc/puppetlabs
[vagrant@client ~]$ sudo puppet apply -e ''
Notice: Compiled catalog for client.example.com in environment production
Notice: Applied catalog in 0.01 seconds

Obviously you won’t be using the vagrant user in a production environment. However, it is not uncommon to see something like the following done to give write access to all members of a certain group:

[vagrant@client ~]$ sudo chgrp -R sysadmin /etc/puppetlabs
[vagrant@client ~]$ sudo chmod -R g+w /etc/puppetlabs

One reason you may not want to change the ownership of this directory would be if your Puppet deployment was managed by continuous integration tools, and you want to ensure that everybody uses the tools. That won’t be a problem when using this book, and chown -R vagrant /etc/puppetlabs is highly recommended for the lessons here.

Running Puppet Without sudo

When you run Puppet with sudo, it uses the system-level configuration files, like so:

[vagrant@client ~]$ sudo puppet config print |grep dir
confdir = /etc/puppetlabs/puppet
codedir = /etc/puppetlabs/code
vardir = /opt/puppetlabs/puppet/cache
logdir = /var/log/puppetlabs/puppet
...etc...

When you run Puppet without sudo, it will use paths in your home directory for configuration files. This can be useful when you are writing and testing small manifests:

[vagrant@client ~]$ puppet config print |grep dir
confdir = /home/vagrant/.puppetlabs/etc/puppet
codedir = /home/vagrant/.puppetlabs/etc/code
vardir = /home/vagrant/.puppetlabs/opt/puppet/cache
logdir = /home/vagrant/.puppetlabs/var/log
...etc...

Most people don’t want to be constantly migrating changes back and forth between two different directory structures. There are two ways to resolve this dilemma:

Use sudo every time you run Puppet

While this is an easy-to-remember way of working, I find myself less comfortable with this option. There are many manifests that can be tested without root access, and many Puppet commands that perform simple lookups for which root access is not required.

Set up a configuration file to use the system-level paths

Modify your personal Puppet configuration to use the system-level paths. This allows you to run Puppet commands that read the same modules and configuration data without invoking root-level access.

I much prefer the second approach, and I recommend it for daily use. For this reason, we have preinstalled the ~/.puppetlabs/etc/puppet/puppet.conf configuration file on all Vagrant configurations, which enables this for you. Check it out:

[vagrant@client ~]$ cat ~/.puppetlabs/etc/puppet/puppet.conf
# Allow "puppet hiera" and "puppet module" commands without sudo
[main]
    logdest = console
    confdir = /etc/puppetlabs/puppet
    codedir = /etc/puppetlabs/code

If you don’t like this setup, you are welcome to remove this file. Just remember to use sudo with every Puppet command, or it will use a different configuration file than you expect.

Running Puppet with sudo

On most systems, the sudo command resets the user search path to a list of known safe directories. Unfortunately, this does not include the Puppet directory. You can see the effect of this change with the following command:

[vagrant@client vagrant]$ env |grep PATH
PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:
  /home/vagrant/.local/bin:/home/vagrant/bin

[vagrant@client vagrant]$ sudo env |grep PATH
PATH=/bin:/sbin:/bin:/usr/sbin:/usr/bin

If you want to run Puppet with sudo, you’ll need to add the /opt/puppetlabs/bin directory to sudo’s secure_path default. The following command will get the current secure_path from the sudoers file, and create a sudoers.d/ config extension that appends /opt/puppetlabs/bin:

$ sudo grep secure_path /etc/sudoers \
    | sed -e 's#$#:/opt/puppetlabs/bin#' \
    | sudo tee /etc/sudoers.d/puppet-securepath
Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/opt/puppetlabs/bin

$ sudo env |grep PATH
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/opt/puppetlabs/bin

Reviewing Puppet Installation

During the course of this chapter, you did the following:

• Enabled the Puppet Yum repository on your system.
• Installed Puppet and its dependencies from this repository.
• Learned the dependencies that support Puppet.
• Reviewed the changed paths used by Puppet 4 on Unix/Linux systems.

Best of all, none of this was done with magic helper scripts that hid the details from you. You can repeat these exact steps on any test or production node to install or upgrade Puppet.

Implementing Resources

Resources are the smallest building block of the Puppet configuration language. They represent a singular element that you wish to evaluate, create, or remove. Puppet comes with many built-in resource types. The stock resource types manipulate system components that you are already familiar with, including:

• Users
• Groups
• Files
• Host file entries
• Packages
• Services

Furthermore, you can create your own resources. However, we’ll begin with one of the simplest resources—the notify resource. Let’s start with the standard first program written in every language:

notify { 'greeting':
  message => 'Hello, world!'
}

This code declares a notify resource with a title of greeting. It has a single attribute, message, which is assigned the value we’d expect in our first program. Attributes are separated from their values by a hash rocket (also called a fat comma), which is a very common way to identify key/value pairs in Perl, Ruby, and PHP scripting languages.

This tiny bit of code is a fully functional and valid manifest. This manifest (with one single resource) does only one thing, which is to output that greeting message every time it is called. Let’s go ahead and use Puppet to evaluate this manifest:

[vagrant@client ~]$ cat /vagrant/manifests/helloworld.pp

notify { 'greeting':
  message => 'Hello, world!'
}

As you can see from this example, manifests are text files named with a .pp file extension—they describe resources using the Puppet configuration language. You can create or modify a Puppet manifest using any text editor.

Applying a Manifest

One of Puppet’s best features is the ease of testing your code. Puppet does not require you to set up complicated testing environments to evaluate Puppet manifests. It is easy—nay, downright trivial—to test a Puppet manifest.

Let’s go ahead and apply this manifest. We will do this using the puppet apply command, which tells Puppet to apply a single Puppet manifest:

[vagrant@client ~]$ puppet apply /vagrant/manifests/helloworld.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: Hello, world!
Notice: /Stage[main]/Main/Notify[greeting]/message:
  defined 'message' as 'Hello, world!'
Notice: Finished catalog run in 0.01 seconds

As you can see, Puppet has applied the manifest. It does this in several steps:

1. Builds (compiles) a Puppet catalog from the manifest.
2. Uses dependency and ordering information to determine evaluation order.
3. Evaluates the target resource to determine if changes should be applied.
4. Creates, modifies, or removes the resource—a notification message is created.
5. Provides verbose feedback about the catalog application.

Don’t worry about memorizing these steps at this point in the learning process. For now, it’s just important that you have a general idea of the process—we’ll discuss these concepts in increasing depth throughout this book. We’ll cover the catalog build and evaluation process in great detail at the end of Part I.

For now, just remember that you’ll use puppet apply to apply Puppet manifests. It will provide you verbose feedback on actions Puppet took to bring the target resources into alignment with the declared policy.

Declaring Resources

There are only a few rules to remember when declaring resources. The format is always the same:

resource_type { 'resource_title':
  ensure     => present,         # usually 'present' or 'absent'
  attribute1 => 1234,            # numbers are unquoted
  attribute2 => 'value',         # strings should be quoted
  attribute3 => ['red','blue'],  # arrays contain other data types
  noop       => false,           # boolean is unquoted true/false
}

The most important rule for resources is: there can be only one. Within a manifest or set of manifests being applied together (the catalog for a node), a resource of a given type can only be declared once with a given title. Every resource of that type must have a unique title.

For example, the following manifest will fail because the same title is used for both file resources:

[vagrant@client ~]$ cat myfile.pp
file { 'my_file':
  ensure => present,
  path   => 'my_file.txt',
}

file { 'my_file':
  ensure => present,
  path   => 'my_file.csv',
}

notify { 'my_file':
  message => "My file is present",
}

[vagrant@client ~]$ puppet apply myfile.pp
Error: Evaluation Error: Error while evaluating a Resource Statement,
  Duplicate declaration: File[my_file] is already declared in file
  /home/vagrant/myfile.pp:1; cannot redeclare at /home/vagrant/myfile.pp:6

You’ll notice that no complaint was given for the notify resource with the same title. This is not a conflict. Only resources of the same type cannot utilize the same title. Naming the preceding files with their full paths ensures no conflicts:

file { '/home/vagrant/my_file.txt':
  ensure => present,
  path   => '/home/vagrant/my_file.txt',
}

file { '/home/vagrant/my_file.csv':
  ensure => present,
  path   => '/home/vagrant/my_file.csv',
}

Viewing Resources

Puppet can show you an existing resource written out in the Puppet configuration language. This makes it easy to generate code based on existing configurations. Let’s demonstrate how to view a resource with an email alias:

[vagrant@client ~]$ puppet resource mailalias postmaster
mailalias { 'postmaster':
  ensure    => 'present',
  recipient => ['root'],
  target    => '/etc/aliases',
}

The puppet resource command queries the resource and shows its current state as Puppet sees it. The output gives you the exact structure, syntax, and attributes to declare this mailalias resource in a Puppet manifest. You can add this to a manifest file, change the recipient, and then use puppet apply to change the postmaster alias on this node.

Let’s examine another resource—the user you are logged in as:

[vagrant@client ~]$ puppet resource user vagrant
Error: Could not run: undefined method 'exists?' for nil:NilClass

This somewhat confusing error message means that you don’t have the privileges to view that resource—so let’s escalate our privileges to complete this command with sudo:

[vagrant@client ~]$ sudo puppet resource user vagrant
user { 'vagrant':
  ensure           => 'present',
  gid              => '1000',
  groups           => ['vagrant'],
  home             => '/home/vagrant',
  password         => '$1$sC3NqLSG$FsXVyW7azpoh76edOfAWm1',
  password_max_age => '99999',
  password_min_age => '0',
  shell            => '/bin/bash',
  uid              => '1000',
}

If you look at the resource, you’ll see why root access was necessary. The user resource contains the user’s password hash, which required root privilege to read from the shadow file. As with the previous mailalias resource, you could write this user resource to a manifest file, replace the password hash, and use sudo puppet apply to change the root password.

Executing Programs

Let’s examine a resource type that executes commands. Use the exec resource to execute programs as part of your manifest:

exec { 'echo-holy-cow':
  path      => ['/bin'],
  cwd       => '/tmp',
  command   => 'echo "holy cow!" > testfile.txt',
  creates   => '/tmp/testfile.txt',
  returns   => [0],
  logoutput => on_failure,
}

Now when you apply this manifest, it will create the testfile.txt file. Notice that we use single quotes to encapsulate the values given to the attributes.

The exec resource declared above uses the creates attribute. This attribute defines a file that will be created by the command execution. When the named file exists, the command is not executed. This means the manifest can be run repeatedly and nothing will change after the file is initially created. Let’s test this out here:

[vagrant@client ~]$ puppet apply /vagrant/manifests/tmp-testfile.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: /Stage[main]/Main/Exec[echo-holy-cow]/returns: executed successfully
Notice: Finished catalog run in 0.07 seconds

[vagrant@client ~]$ puppet apply /vagrant/manifests/tmp-testfile.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: Finished catalog run in 0.01 seconds

There are a wide variety of attributes you can use to control whether or not an exec resource will be executed, and which exit codes indicate success or failure. This is a complex and feature-rich resource. Whenever you create an exec resource, test carefully and refer to “Puppet Type Reference: exec” on the Puppet docs site.

Managing Files

How else could we create this file? We could have used the file resource. Let’s examine one now:

file { '/tmp/testfile.txt':
  ensure  => present,
  mode    => '0644',
  replace => true,
  content => 'holy cow!',
}

This is a proper declarative policy. We declare that the file should exist, and what the contents of the file should be. We do not need to concern ourselves with how, or when to make changes to the file. Furthermore, we were able to ensure the contents of the file remained consistent, which is not possible within an echo command.

Let’s apply this policy now:

[vagrant@client ~]$ puppet apply /vagrant/manifests/file-testfile.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: /Stage[main]/Main/File[/tmp/testfile.txt]/content: content changed
  '{md5}0eb429526e5e170cd9ed4f84c24e4' to '{md5}3d508c8566858d8a168a290dd709c'
Notice: /Stage[main]/Main/File[/tmp/testfile.txt]/mode:
  mode changed '0664' to '0644'
Notice: Finished catalog run in 0.03 seconds

[vagrant@client ~]$ puppet apply /vagrant/manifests/file-testfile.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: Finished catalog run in 0.02 seconds

Unlike with the previous exec resource, Puppet observed that the contents were different and changed the file to match. Now, you’re probably thinking to yourself, “Aren’t the file contents the same in both?” Nope. It’s not obvious in the exec declaration, but echo appends a trailing newline to the text. As you can see here, the file contents don’t include a newline:

[vagrant@client ~]$ cat /tmp/testfile.txt
holy cow![vagrant@client ~]$

You can easily adjust the file contents to include as many newline characters as you want. Because the newline character is interpolated, you’ll need to use double quotes around the contents.

You could also change the replace attribute from true to false if you want to initially create a missing file, but not replace one that has been changed.

file { '/tmp/testfile.txt':
  ensure  => present,
  mode    => '0644',
  replace => false,
  content => "holy cow!\n",
}

You can find complete details of the many attributes available for the file resource at “Puppet Type Resource: file” on the Puppet docs site.

$ sudo puppet filebucket --local backup /tmp/testfile.txt
/tmp/testfile.txt: 3d508c856685853ed8a168a290dd709c

$ sudo puppet filebucket --local list
0eb429526e5e170cd9ed4f84c24e442b 2015-11-19 08:18:06 /tmp/testfile.txt
3d508c856685853ed8a168a290dd709c 2015-11-19 08:23:42 /tmp/testfile.txt

$ sudo puppet filebucket --local get 0eb429526e5e170cd9ed4f84c24e442b
holy cow!

$ sudo puppet filebucket --local diff \
    0eb429526e5e170cd9ed4f84c24e442b /tmp/testfile.txt
--- /tmp/diff20151119-4940-1h3dwn9	2015-11-19 08:20:59.994974403 +0000
+++ /tmp/testfile.txt	2015-11-19 08:18:06.162104809 +0000
@@ -1 +1 @@
-holy cow!
+holy cow!
\ No newline at end of file

$ sudo puppet filebucket --local restore \
    /tmp/testfile.old 0eb429526e5e170cd9ed4f84c24e442b

Avoiding Imperative Manifests

In the previous section, we established that it is best practice to avoid using exec resources. To understand the reason for this, let’s return to our previous discussion of the word declarative.

If the code tells the interpreter what to do, when to do it, and how to do it, then it is functioning in an imperative manner. If the code tells the interpreter what it wants the result to be, then the code is declarative.

Yes, the exec resource will let you write imperative manifests in Puppet. However, as we discussed in Chapter 1, you’ll find that it takes more effort, is harder to maintain, and is less likely to produce consistent results.

Rob Nelson provided us with the perfect metaphor:¹

If you try hard enough, you can make it act like an imperative language. That would be akin to ripping the bottom out of your car, dressing in a leopard-print toga and a ratty blue tie, and driving to work with your feet. Is that something you really want to do, or just watch on Boomerang?

The reason I advocate so strongly to not use exec is fairly simple. The exec resource executes the command, but it doesn’t really know what the command does. That command could modify the node state in any form, and Puppet wouldn’t be aware of it. This creates a fire and forget situation, where the only piece of information returned to Puppet is the exit code from the command:

Notice: /Stage[main]/Main/Exec[echo-holy-cow]/returns: executed successfully

All you know is that the command here returned an exit code you expected (0 by default, which is the Unix convention for success). You don’t really know what the command did. If someone wrote a Puppet manifest last week, and then removed it from the execution this week, you have to compare backups (if you have any) to determine what the command did.

When you declare resources that specify desired state, Puppet logs both the previous and the changed values. Files that are changed are backed up, and can be restored if necessary. For example, let’s examine what happened when we applied the file resource:

Notice: /Stage[main]/Main/File[/tmp/testfile.txt]/content: content changed
  '{md5}0eb429526e5e170cd9ed4f84c24e44' to '{md5}3d508c856685853ed8a168a290dd70'
Notice: /Stage[main]/Main/File[/tmp/testfile.txt]/mode:
  mode changed '0664' to '0644'

Without having access to read the original Puppet manifest, you know that the file mode and the file contents were changed. You can validate the file contents later to match the md5 hash, or restore a copy of the file as it was before this change was made.

Using the file resource is more concise, easier to read, and more consistent in application. Best of all, it creates a detailed log of which changes were made. I have found this is true of every situation where you can replace an exec resource with a native resource. Use an exec resource only when no other choice is available.

Testing Yourself

Let’s pause for a second and utilize what you have learned to build a Puppet manifest:

1. Use puppet resource to create a new manifest from the file /tmp/testfile.txt.
2. Remove the type attribute, which is redundant with the resource name.
3. Change the file mode to be group writable (either 0664 or ug=rw,o=r).
4. Change the content to say something that amuses you.
5. Run puppet apply yourmanifest.pp and observe the changes.
6. Confirm the changes with the following commands:
   • ls -la /tmp
   • cat /tmp/testfile.txt
7. Run puppet apply again and see what happens.

You won’t need to use sudo to make any of these changes, as you are the owner of this file.

Reviewing Writing Manifests

In this chapter, you learned about manifests, single files that contain Puppet resources to be evaluated and, if necessary, applied on the node. Manifests are the closest thing that could be called a program in the conventional sense.

You learned how to create and apply resources, the smallest building blocks of a Puppet manifest. You learned the common syntax of a resource, and how to retrieve the details of an existing resource on a system in the Puppet configuration language.

In addition, you learned how to instruct the Puppet agent to output messages during application of the manifest for debugging purposes.

Finally, you learned how to execute programs with an exec resource, as well as why this can be bad design. You learned how a more declarative style requires less code, and works more consistently in all situations.

Defining Variables

Puppet makes use of variables (named storage of data) much like any other language you’ve learned to write code in.

Like many scripting languages, variables are prefaced with a $ in Puppet. The variable name must start with a lowercase letter or underscore, and may contain lowercase letters, numbers, and underscores. Let’s take a look at a few examples of both valid and invalid variable names:

$myvar         # valid
$MyVar         # invalid

$my_var        # valid
$my-var        # invalid

$my3numbers    # valid
$3numbers      # invalid

Variables starting with underscores should only be used within the local scope (we’ll cover variable scope in “Understanding Variable Scope”):

$_myvar        # valid inside a local scope
$prog::_myvar  # deprecated: no underscore-prefixed variables out of scope

Variables are assigned values with an equals sign. Every value has a data type. The most common types are Boolean, Numeric, and String. As I’m sure you’ve seen these data types many times before, we’ll jump straight to some examples:

$my_name       = 'Jo'         # string
$num_tokens    = 115          # numeric
$not_true      = false        # boolean

A variable that has not been initialized will evaluate as undefined, or undef. You can also explicitly assign undef to a variable:

$notdefined = undef

The puppetlabs/stdlib module provides a function you can use to see a variable’s type:

include stdlib
$nametype = type_of( $my_name )     # String
$numtype  = type_of( $num_tokens )  # Integer

$decimal     = 1234      # valid Integer decimal assignment
$float       = 12.34     # valid Float decimal assignment
$octal       = 0775      # valid Integer octal assignment
$hexadecimal = 0xFFAA    # valid Integer hexadecimal assignment
$string      = '001234'  # string containing a number with leading zeros

$my_list  = [1,3,5,7,11]          # array of Numeric values
$my_names = ['Amy','Sam','Jen']   # array of String values
$mixed_up = ['Alice',3,true]      # String, Number, Boolean
$trailing = [4,5,6,]              # trailing commas OK, unlike JSON
$embedded = [4,5,['a','b']]       # number, number, array of strings

[$first,$middle,$last] = ['Jo',undef,'Rhett']  # good
[$first,$middle,$last] = ['Jo','Rhett']        # error

myfunction( *$Array_of_arguments ) { ... }

# small hash of user home directories
$homes = { 'jack' => '/home/jack', 'jill' => '/home/jill', }

# Multiline definition
$user = {
  'username' => 'jill',  # String value
  'uid'      => 1001,    # Integer value
  'create'   => true,    # Boolean value
}

$my_name = 'Dr. Evil'
$how_much = '100 million'

notice( "Hello ${username}, glad to see you today!" )

$message_text = @(END)
This is a very long message,
which will be composed over
many lines of text.
END

$message_text = @("END")
Dear ${user},
  Your password is ${password}.

  Please login at ${site_url} to continue.
END

$my_passphrase = Sensitive('One million dollars')

$ puppet apply -e "notice(Sensitive('One million dollars'))"
Notice: Scope(Class[main]): Sensitive [value redacted]

$the_greeting = "Hello ${myname}, you have received ${num_tokens} tokens!"

notice( "The second value in the list is ${my_list[1]}" )

notice( "The second value in the list is $my_list[1]" )

# Output value from an array index 1
notice( "The second value in the list is ${my_list[1]}" )

# Output value stored in hash key alpha
notice( "The value stored with key alpha is ${my_hash['alpha']}" )

# This time we define the strings in advance
$file_name = "/tmp/testfile2-${my_name}.txt"
$the_greeting = "Hello ${myname}, you have received ${num_tokens} tokens!"

# Don't use braces for variables that stand alone
file { $file_name:
  ensure  => present,
  mode    => '0644',
  replace => true,
  content => $the_greeting,
}

# Work around the need for both types of quotes in a variable
$the_greeting = "we need 'single quotes' and \"double quotes\" here."

# Place backslashes before special characters to avoid interpolation
$describe = "\$user uses a \$ dollar sign and a \\ backslash"

# Previously interpolated values won't be interpolated again
$inform = "${describe}, and resolves to the value '${user}'."

$num_tokens = '$100 million'    # dollars, not a variable
$cifs_share = '\\server\drive'  # windows share, not escape chars

myvariable = 10
print myvariable  # prints 10
myvariable = 20
print myvariable  # prints 20

[vagrant@client ~]$ cat double-assign.pp 
$myvar = 5
$myvar = 10

[vagrant@client ~]$ puppet apply double-assign.pp
Error: Cannot reassign variable myvar at double-assign.pp:2

$my_variable = somestring    # valid but risky
$my_variable = 'somestring'  # best practice

Finding Facts

Speaking of variables, Facter provides many variables for you containing node-specific information. These are always available for use in your manifests. Go ahead and run facter and look at the output:

[vagrant@client ~]$ facter
architecture => x86_64
augeasversion => 1.0.0
blockdevice_sda_model => VBOX HARDDISK
blockdevice_sda_size => 10632560640
blockdevice_sda_vendor => ATA
blockdevices => sda
domain => example.com
facterversion => 3.1.2
filesystems => ext4,iso9660
fqdn => client.example.com
gid => vagrant
hardwareisa => x86_64
hardwaremodel => x86_64
hostname => client
id => vagrant
interfaces => eth0,eth1,lo
ipaddress => 10.0.2.15
...etc

As you can see, Facter produces a significant number of useful facts about the system, from facts that won’t change over the lifetime of a system (e.g., platform and architecture) to information that can change from moment to moment (e.g., free memory). Try the following commands to find some of the more variable fact information provided:

[vagrant@client ~]$ facter | grep version

[vagrant@client ~]$ facter | grep mb

[vagrant@client ~]$ facter | grep free

Puppet adds several facts for use within Puppet modules. You can run the following command to see all facts used by Puppet, and installed on the node from Puppet modules:

[vagrant@client ~]$ facter --puppet

Puppet always adds the following facts above and beyond system facts provided by Facter:

$facts['clientcert']

The client-reported value of the node’s certname configuration value.

$facts['clientversion']

The client-reported version of Puppet running on the node.

$facts['clientnoop']

The client-reported value of whether noop was enabled in the configuration or on the command line to perform the comparison without actually making changes.

$facts['agent_specified_environment']

The environment requested by the client. When using a Puppet server, the server could override the environment selection. This value contains the requested value. If this value is blank, the production environment is used by default.

All of these facts can be found in the $facts hash.

You can also use Puppet to list out Puppet facts in JSON format:

[vagrant@client ~]$ puppet facts find
$ puppet facts find
{
  "name": "client.example.com",
  "values": {
    "puppetversion": "4.10.9",
    "virtual": "virtualbox",
    "is_virtual": true,
    "architecture": "x86_64",
    "augeasversion": "1.4.0",
    "kernel": "Linux",
    "domain": "example.com",
    "hardwaremodel": "x86_64",
    "operatingsystem": "CentOS",

Facter can provide the data in different formats, useful for passing to other programs. The following options output Facter data in the common YAML and JSON formats:

[vagrant@client ~]$ facter --yaml
[vagrant@client ~]$ puppet facts --render-as yaml

[vagrant@client ~]$ facter --json
[vagrant@client ~]$ puppet facts --render-as json

Calling Functions in Manifests

A function is executable code that may accept input parameters, and may output a return value. A function that returns a value can be used to provide a value to a variable:

$zero_or_one = bool2num( $facts['is_virtual'] );

The function can also be used in place of a value, or interpolated into a string:

# md5() function provides the value for the message attribute
notify { 'md5_hash':
  message => md5( $facts['fqdn'] ),
}

# Include the MD5 hash in the result string
$result = "The MD5 hash for the node name is ${md5( $facts['fqdn'] )}"

Functions can also take action without returning a value. Previous examples used the notice() function, which sends a message at notice log level but does not return a value. In fact, there is a function for logging a message at each level, including all of the following (shown in increasing severity):

• debug( message )
• info( message )
• notice( message )
• warning( message )
• err( message )
• crit( message )
• alert( message )
• emerg( message )

Puppet executes functions when building the catalog; thus, functions can change the catalog. Some of the more common uses for this level of power are:

• Look up data from external sources
• Add, modify, or remove items from the catalog
• Dynamically generate and execute code segments

Functions can be written in the common prefix format or in the Ruby-style chained format. The following two calls will return the same result:

# Common prefix format
notice( 'this' )

# Ruby-style chained format
'this'.notice()

As always, use the form that is easier to read where it is used in the code.

Using Variables in Resources

Now let’s cover how to use variables in resources. Each data type has different methods, and sometimes different rules, about how to access its values.

Constant strings without variables in them should be surrounded by single quotes. Strings containing variables to be interpolated should be surrounded by double quotes. No other type should be quoted. Here’s an example:

notice( 'Beginning the program.' )
notice( "Hello, ${username}" )
notice( 1000000 )
notice( true )

You can access specific items within an Array by using a 0-based array index within square brackets. Two indices can be specified to retrieve a range of items:

$first_item = $my_list[1]
$four_items = $my_list[3,6]

You can access specific items within a Hash by using the hash key within square brackets as follows:

$username = $my_hash['username']

Use curly braces when interpolating variables into a double-quoted string. The curly braces must surround both the variable name and the index or key (within square brackets) when accessing hash keys or array indexes:

notice( "The user's name is ${username}" )
notice( "The second value in my list is ${my_list[1]}" )
notice( "The login username is ${my_hash['username']}" )

Curly braces are only necessary when you’re interpolating a variable within a string. Do not use braces or quotes when using the variable by itself. Here is an example of using predefined variables properly:

file { $filename:
  ensure  => present,
  mode    => '0644',
  replace => $replace_bool,
  content => $file['content'],
}

Retrieve specific values from a Hash by assigning to an Array of variables named for the keys you’d like to retrieve. Read that sentence carefully—the name of the variable in the array identifies the hash key to get the value from:

[$jack] = $homes          # identical to $jack = $homes['jack']
[$username,$uid] = $user  # gets the values assigned to keys "username" and "uid"
$jill = $user             # oops, got the entire Hash!

The facts provided by Facter are available in a $facts hash. For example, to customize the message shown on login to each node, use a file resource like this:

file { '/etc/motd':
  ensure  => present,
  mode    => '0644',
  replace => true,
  content => "${facts['hostname']} runs ${facts['os']['release']['full']}",
}

Older Puppet manifests refer to facts using just the fact name as a variable, such as $factname. This is dangerous, as the fact could be overwritten either deliberately or accidentally within the scope in which the code is operating. A slight improvement is to refer to the fact explicitly in the top scope with $::factname. However, the variable could be overridden there as well. Finally, neither of these options informs a code reviewer whether the value was defined in a manifest, or by a fact.

You can receive Evaluation Error exceptions when Puppet tries to use a variable that has never been defined by enabling the strict_variables configuration setting in /etc/puppetlabs/puppet/puppet.conf:

[main]
  strict_variables = true

This will not cause an error when a variable has been explicitly set to undef. It will only throw an exception if the variable has never been declared:

$ puppet apply --strict_variables /vagrant/manifests/undefined.pp
Notice: Scope(Class[main]):
Error: Evaluation Error: Unknown variable: 'never_defined'.
  at /vagrant/manifests/undefined.pp:4:9 on node client.example.com

Defining Attributes with a Hash

It is also possible to pass a hash of names and values as default values in a resource declaration. Use attribute name of * (called a splat) with a value of the hash containing defaults.

Here’s an example:

$resource_attributes = {
  ensure    => present,
  owner     => 'root',
  group     => 'root',
  'mode'    => '0644',
  'replace' => true,
}

file { '/etc/config/first.cfg':
  source => 'first.cfg',
  *      => $resource_attributes,
}

file { '/etc/config/second.cfg':
  source => 'config.cfg',
  *      => $resource_attributes,
}

The splat operator allows you to share default values across multiple resource declarations. This is an essential strategy for don’t repeat yourself (DRY) development.

Declaring Multiple Resource Titles

It is possible to declare multiple resources within a single declaration. The first way you can do so is by supplying an array of titles with the same resource body. The following example works because the file’s name defaults to the title if not supplied as an attribute:

file { ['/tmp/file_one.txt','/tmp/file_two.txt']:
  ensure => present,
  owner  => 'vagrant',
}

This definition creates two different file resources in the same declaration by providing two different titles. This only works successfully when the title is reused as a parameter that makes the resource unique.

Declaring Multiple Resource Bodies

The title and attributes of a resource are called the resource body. A single resource declaration can have multiple resource bodies separated by semicolons. Here’s an example of two file resources within a single resource declaration:

file {
  'file_one':
    ensure => present,
    owner  => 'vagrant',
    path   => 'file_one.txt',
  ;

  'file_two':
    ensure => present,
    owner  => 'vagrant',
    path   => 'file_two.txt',
  ;
}

This format would create two resources, exactly as if these were done in two different resource declarations. As this is not necessarily easier to read, the single resource per declaration is considered a better practice, except when using default values.

If one of the resource bodies has the title default, it is not used to create a resource. Instead, it defines defaults for the other resource bodies. Here is an example where the preceding definition becomes shorter and easier to maintain:

file {
  default:
    ensure => present,
    owner  => 'vagrant',
  ;

  'file_one': path => 'file_one.txt';
  'file_two': path => 'file_two.txt';
}

Any of the resources can provide an override for an attribute, in which case the default value is ignored.

Modifying with Operators

You can use all of the standard arithmetic operators for variable assignment or evaluation. As before, we’re going to provide examples and skip an explanation you’ve likely gotten many times in your life:

$added       = 10 + 5       # 15
$subtracted  = 10 - 5       # 5
$multiplied  = 10 * 5       # 50
$divided     = 10 / 5       # 2
$remainder   = 10 % 5       # 0
$two_bits_l  = 2 << 2       # 8
$two_bits_r  = 64 >> 2      # 16

We’ll cover the comparison operators in the next section about conditionals.

$my_list = [1,4,7]
$bigger_list = $my_list + [14,17]   # equals [1,4,7,14,17]

$key_pairs = {name => 'Joe', uid => 1001}
$user_definition = $key_pairs + { gid => 500 }  # hash now has name, uid, gid...

$longer_list = $my_list << 33      # equals [1,4,7,33]
$unintended = $my_list << [33,35]  # equals [1,4,7,[33,35]]

# Remove a single value
$names = ['jill','james','sally','sam','tigger']
$no_tigger = $names - 'tigger'

# Remove multiple values
$no_boys = $names - ['james','sam']

# Remove a single key
$user = {name => 'Jo', uid => 1001, gid => 500 }
$no_name = $user - 'name'

# Remove multiple keys
$user = {name => 'Jo', uid => 1001, gid => 500 }
$only_name = $user - ['uid','gid']

# Remove all matching keys from another hash
$compare = {name => 'Jo', uid => 1001, home => '/home/jo' }
$difference = $user - $compare

# you don't need to know operator precedence to understand this
$myvar = 5 * (10 + $my_var)

Using Comparison Operators

If you have any experience programming, you’ll find Puppet’s comparison operators familiar and easy to understand. Any expression using comparison operations will evaluate to boolean true or false. First, let’s discuss all the ways to evaluate statements. Then we’ll review how to use the boolean results.

Number comparisons operate much as you might expect:

4 != 4.1                 # number comparisons are simple equality match
$how_many_cups < 4       # any number less than 4.0 is true
$how_many_cups >= 3      # any number larger than or equal to 3.0 is true

String operators are a bit inconsistent. String and substring equality comparisons are case insensitive for US ASCII characters (0-9, a-z, etc), as shown here:

 coffee  == 'coffee'     # bare word string is equivalent to quoted single word
'Coffee' == 'coffee'           # string comparisons are case insensitive
!('tea' in 'coffee')           # you can't find tea in coffee
!('Fee' in 'coffee')           # substring matches are case insensitive

UTF-8 characters in strings only match exact case. String comparison is done on a character-by-character basis, so a UTF-8 string could match a partial based on mixed-case ASCII characters. I know this is confusing, so here are some examples:

("Touch"  in "touch\u00e9")    # compared only us-ascii characters
!("\u00c9"  in "touch\u00e9")  # No É in touché as UTF8 accent characters 
                               # are case sensitive

Array and hash comparisons match only with complete equality of both length and value. The in comparison looks for value matches in arrays, and key matches in hashes:

[1,2,5] != [1,2]                   # array matching tests for identical arrays
5 in [1,2,5]                       # value found in array

{name => 'Joe'} != {name => 'Jo'}           # hashes aren't identical
'Jo' !in {fname => 'Jo', lname => 'Rhett'}  # Jo is a value and doesn't match 

You can also compare values to data types, like so:

$not_true =~ Boolean        # true if true or false
$num_tokens =~ Integer      # true if an integer
$my_name !~ String          # true if not a string

As this feature has the most benefit for input validation in Puppet modules, we cover this topic extensively in “Validating Input with Data Types”. For now, just be aware that it is possible.

When doing comparisons you’ll find the standard boolean operators and, or, and ! (not) work exactly as you might expect:

true and true               # true
true and false              # false
true or false               # true
true and !false             # true
true and !true              # false

You can find a complete list of all operands and operators with example uses at “Language: Expressions and Operators” on the Puppet docs site.

Evaluating Conditional Expressions

Now, let’s use these expressions you’ve learned with conditional statements. You have four different ways to utilize the boolean results of a comparison:

• if/elsif/else statements
• unless/else statements
• case statements
• Selectors

As you’d expect, there’s always the basic conditional form I’m sure you know and love:

if ($coffee != 'drunk') {
  notify { 'best-to-avoid': }
}
elsif ('scotch' == 'drunk') {
  notify { 'party-time': }
}
else {
  notify { 'party-time': }
}

There’s also unless to reverse comparisons for readability purposes:

unless $facts['kernel'] == Linux {
  notify { 'You are on an older machine.': }
}
else {
  notify { 'We got you covered.': }
}

While unless is considered bad form by some, I recommend sticking with the most readable form. The following example shows why unless can be tricky reading with an else clause:

# The $id fact tells us who is running the Puppet agent
unless( $facts['id'] == 'root' ) {
  notify { 'needsroot':
    message => "This manifest must be executed as root.",
  }
}
else {
  notify { 'isroot':
    message => "Running as root.",
  }
}

The case operator can be used to do numerous evaluations, avoiding a long string of multiple elsif(s). You can test explicit values, match against another variable, use regular expressions, or evaluate the results of a function. The first successful match will execute the code within the block following a colon:

case $what_she_drank {
  'wine':            { include state::california }
  $stumptown:        { include state::portland   }
  /(scotch|whisky)/: { include state::scotland   }
  is_tea( $drink ):  { include state::england    }
  default:           {} 
}

Always include a default: option when using case statements, even if the default does nothing, as shown in the preceding example.

Statements with selectors are similar to case statements, except they return a value instead of executing a block of code. This can be useful when you are defining variables. A selector looks like a normal assignment, but the value to be compared is followed by a question mark and a block of comparisons with fat commas identifying the matching values:

$native_of = $what_he_drinks ? {
  'wine'            => 'california',
  $stumptown        => 'portland',
  /(scotch|whisky)/ => 'scotland',
  is_tea( $drink )  => 'england',
  default           => 'unknown',
}

As a value must be returned in an assignment operation, a match is required. Always include a bare word default option with a value.

So the drinking comparisons have been fun, but let’s examine some practical comparisons that you may actually use in a real manifest. Here’s a long if/then/else chain:

# Explicit comparison
if( $facts['os']['family'] == 'redhat' ) {
  include yum
}
# Do a substring match
elsif( $facts['os']['family'] in 'debian-ubuntu' ) {
  include apt
}
# New package manager available with FreeBSD 9 and above
elsif( $facts['operatingsystem'] =~ /?i:freebsd/ )
      and ( Integer($facts['os']['release']['major']) >= 9 ) {
  include pkgng
}

The same result can be had in a more compact case statement:

case $facts['os']['family'] {
  'redhat':                                                       { include yum   }
  'debian', 'ubuntu':                                             { include apt   }
  'freebsd' and ( Integer($facts['os']['release']['major']) >= 9 ) { include pkgng }
  default: {}
}

Selectors are also useful for handling heterogenous environments:

$libdir = $facts['os']['family'] ? {
  /(?i-mx:centos|fedora|redhat)/ => '/usr/libexec/mcollective',
  /(?i-mx:ubuntu|debian)/        => '/usr/share/mcollective/plugins',
  /(?i-mx:freebsd)/              => '/usr/local/share',
}

The splat operator ( *) can turn an array of values into a list of choices. This can be very useful in case or select statements, as shown here:

$redhat_based = ['RedHat','Fedora','CentOS','Scientific','Oracle','Amazon']
$libdir = $facts['os']['family'] ? {
  *$redhat_based => '/usr/libexec/mcollective',

You can find a complete list of all conditional statements with more example uses at “Language: Conditional Statements and Expressions” on the Puppet docs site.

Matching Regular Expressions

Puppet supports standard Ruby regular expressions, as defined in the Ruby Regexp docs. The match operator (=~) requires a String value on the left, and a Regexp expression on the right:

$what_did_you_drink =~ /tea/        # likely true if English
$what_did_you_drink !~ /coffee/     # likely false if up late
$what_did_you_drink !~ "^coffee$"   # uses a string value for regexp

The value on the left must be a string. The value on the right can be a /Regexp/ definition within slashes, or a string value within double quotes. The use of a string allows variable interpolation to be performed prior to conversion into a Regexp.

You can use regular expressions in four places:

• Conditional statements: if and unless
• case statements
• Selectors
• Node definitions (deprecated)

As regular expressions are well documented in numerous places, we won’t spend time covering how to use them here, other than to provide some examples:

unless $facts['operatingsystem'] !~ /(?i-mx:centos|fedora|redhat)/ {
  include yum
}

case $facts['hostname'] {
  /^web\d/: { include role::webserver  }
  /^mail/ : { include role::mailserver }
  default : { include role::base       }
}

$package_name = $facts['operatingsystem'] ? {
  /(?i-mx:centos|fedora|redhat)/ => 'mcollective',
  /(?i-mx:ubuntu|debian)/        => 'mcollective',
  /(?i-mx:freebsd)/              => 'sysutils/mcollective',
}

You may find Tony Stubblebine’s Regular Expressions Pocket Reference (O’Reilly) handy for day-to-day work with Regexps. To truly master regular expressions, there is no better book than, Jeffrey E.F. Friedl’s Mastering Regular Expressions (O’Reilly).

Building Lambda Blocks

A lambda is a block of code that allows parameters to be passed in. You can think of them as functions without a name. You will use lambdas with the iterator functions (such as each(), introduced in the next section) to evaluate values with a block of code. If you are experienced with Ruby lambdas, you’ll find the syntax similar.

A lambda begins with one or more variable names between pipe operators | |. These variables will receive the values passed into the block of code:

| $firstvalue, $secondvalue | {
  block of code that operates on these values.
}

The lambda has its own variable scope. This means that the variables named between the pipes exist only within the block of code. You can name these variables any name you want, as they will be filled by the values passed by the function into the lambda on each iteration. Other variables within the context of the lambda are also available, such as local variables or node facts.

The following example will output a list of disk partitions from the hash provided by Facter. Within the loop, we refer to the hostname fact on each iteration. The device name and a hash of values about each device are stored in the $name and $device variables during each loop:

$ cat /vagrant/manifests/mountpoints.pp
each( $facts['partitions'] ) |$name, $device| {
  notice( "${facts['hostname']} has device ${name} with size ${device['size']}" )
}

$ puppet apply /vagrant/manifests/mountpoints.pp
Notice: Scope(Class[main]): Host geode has device sda1 with size 524288
Notice: Scope(Class[main]): Host geode has device sda2 with size 3906502656
Notice: Scope(Class[main]): Host geode has device sdb1 with size 524288
Notice: Scope(Class[main]): Host geode has device sdb2 with size 3906502656

Next, we’ll cover each of the functions that can iterate over values and pass them to a lambda.

Looping Through Iterations

In this section, we’re going to introduce powerful new functions for iterating over sets of data. You can use iteration to evaluate many items within an array or hash of data using a single block of code (a lambda, described on the previous page).

Here are some practical examples available to you from the basic facts provided by Facter (you can use iteration with any data point that can be presented as an array or a hash):

• Find all interfaces assigned IP addresses from the RFC-1918 space.
• Count the number of processor cores on the node.
• Total up free space on all mounted partitions.

There are five functions that iterate over a set of values and pass each one to a lambda for processing. The lambda will process each input and return a single response containing the processed values. Here are the five functions, what they do to provide input to the lambda, and what they expect the lambda to return as a response:

• each() acts on each entry in an array, or each key/value pair in a hash.
• filter() returns a subset of the array or hash that were matched by the lambda.
• map() returns a new array or hash from the results of the lambda.
• reduce() combines array or hash values using code in the lambda.
• slice() creates small chunks of an array or hash and passes it to the lambda.

The following examples show how these functions can be invoked. They can be invoked in traditional prefix style:

each( $facts['partitions'] ) |$name, $device| {
  notice( "${facts['hostname']} has device $name with size ${device['size']}" )
}

Or you can chain function calls to the values they operate on, which is a common usage within Ruby:

$facts['partitions'].each() |$name, $device| {
  notice( "${facts['hostname']} has device $name with size ${device['size']}" )
}

Finally, new to Puppet 4 is the ability to use a hash or array literal instead of a variable. The following example demonstrates iteration over a literal array of names:

['sally','joe','nancy','kevin'].each() | $name | {
  notice( "$name wants to learn more about Puppet." )
}

Now let’s review each of the functions that can utilize a lambda.

# Output a list of interfaces that have IPs
split( $facts['interfaces'], ',' ).each |$interface| { 
  if( $facts["ipaddress_${interface}"] != '' ) {
    notice( sprintf( "Interface %s has IPv4 address %s",
      $interface, $facts["ipaddress_${interface}"] )
    )
  }
  if( $facts["ipaddress6_${interface}"] != '' ) {
    notice( sprintf( "Interface %s has IPv6 address %s",
      $interface, $facts["ipaddress6_${interface}"] )
    )
  }
}

[vagrant@client puppet]$ puppet apply /vagrant/manifests/interface_ips.pp
Notice: Scope(Class[main]): Interface enp0s3 has IPv4 address 10.0.2.15
Notice: Scope(Class[main]): Interface enp0s3 has IPv6 address fe80::a0:27:feb:2b2
Notice: Scope(Class[main]): Interface enp0s8 has IPv4 address 192.168.250.10
Notice: Scope(Class[main]): Interface enp0s8 has IPv6 address fe80::a0:27:fec:d78
Notice: Scope(Class[main]): Interface lo has IPv4 address 127.0.0.1
Notice: Scope(Class[main]): Interface lo has IPv6 address ::1

$ cat /vagrant/manifests/interfaces.pp
split( $facts['interfaces'], ',' ).each |$index, $interface| { 
  notice( "Interface #${index} is ${interface}" )
}

$ puppet apply /vagrant/manifests/interfaces.pp
Notice: Scope(Class[main]): Interface #0 is enp0s3
Notice: Scope(Class[main]): Interface #1 is enp0s8
Notice: Scope(Class[main]): Interface #2 is lo

$ cat /vagrant/manifests/uptime.pp
each( $facts['system_uptime'] ) |$type, $value| {
  notice( "System has been up ${value} ${type}" )
}

$ puppet apply /vagrant/manifests/uptime.pp
Notice: Scope(Class[main]): System has been up 23:04 hours uptime
Notice: Scope(Class[main]): System has been up 83044 seconds
Notice: Scope(Class[main]): System has been up 23 hours
Notice: Scope(Class[main]): System has been up 0 days

each( $facts['system_uptime'] ) |$uptime| {
  notice( "System has been up ${uptime[1]} ${uptime[0]}" )
}

$ puppet apply /vagrant/manifests/ipaddresses.pp
Notice: Scope(Class[main]): interface enp0s3 has private IP fe80::a0:27:feb:2b28
Notice: Scope(Class[main]): interface enp0s8 has private IP fe80::a0:27:fec:d78c
Notice: Scope(Class[main]): interface enp0s3 has private IP 10.0.2.15
Notice: Scope(Class[main]): interface enp0s8 has private IP 192.168.250.10