Trusted Facts

When a node connects to the server it transmits a list of facts about the node. Unfortunately, there is no way to validate that these facts are correct. If you have conditions in your code based on node information that provide access to security-related data, the data could be accessed by a compromised node that forged its facts.

There is a new hash available named $trusted[]. It contains facts that have been validated by the server, and can be trusted to be correct.

$trusted['authenticated']

The possible values are:

remote

Confirms a successful validation of the remote client’s certificate

local

Indicates that puppet apply is being used and validation is unavailable

false

Warns that the auth.conf file is misconfigured to allow unauthenticated requests

$trusted['certname']

This contains the certificate name validated by the Puppet server if remote, or the value of $certname in the Puppet configuration if local.

$trusted['hostname'] and $trusted['domain']

hostname will contain the part before the first period of the certificate name, and domain will contain the remainder. This is useful when you’re using FQDN certnames, and confusing in other situations.

$trusted['extensions']

The value of this will be a hash containing all certificate extensions present in the certificate. You can find out how to create certificate extensions in “Policy-Based Autosigning”.

You can significantly improve security by altering the Hiera hierarchy to use the trusted[] facts provided by the server instead of the facts[] supplied by the client:

:hierarchy:
  - "nodes/%{trusted.certname}"
  - "os/%{facts.os.family}"
  - common

More values could be added to Trusted Facts by later releases of Puppet. You can see the current list at “Facts and Built-in Variables: Trusted Facts” on the Puppet docs site.

Server Facts

The server sets certain variables for use in manifests. Unfortunately, if the node provides facts of the same name, these values will be overridden. If you enable the following setting, the Puppet server will define a new hash of server facts that a client cannot override:

[master]
   trusted_server_facts = true

When this value is enabled (and the Puppet server reloaded), there will be a new hash available named $server_facts[] that provides the following values:

$server_facts['serverversion']

Version of Puppet used by the server. Not the version of Puppet Server!

$server_facts['servername']

The certificate name (certname) of the server. The name used to sign all certificates if the server provides the CA function.

$server_facts['serverip']

The IP address it was contacted on. This can vary on multihomed servers.

$server_facts['environment']

The environment selected by the server for the node. It is possible for the node to request one environment from its configuration files, or command-line arguments, but provide a fact that sets the top-level variable to another value. It won’t be the value used when building the catalog, which could be confusing. This entry in the server_facts[] hash will always contain the environment used for processing the node’s catalog.

More server facts could be added by later releases of Puppet. You can see the current list at “Language: Facts and Build-in Variables” on the Puppet docs site.

Server Configuration Settings

As shown in the following example, Puppet servers make every configuration setting available as $::setting::setting_name:

notice("Trusted server facts are enabled: ${::settings::trusted_server_facts}")

Note that this is not a hash but a namespace, so you cannot iterate over it to output each value.

There is no equivalent access to node configuration settings by default, other than the following:

• trusted['certname']
• facts['clientnoop']
• facts['agent_specified_environment']

To gain access to a node’s configuration values, add a custom fact or external fact to a Puppet module, as documented in “Adding Custom Facts”.

Enabling Transmission of Reports

Agents communicating with a Puppet server send a node convergence report to the same server from which they received the catalog. You can change this configuration with the following parameters in /etc/puppetlabs/puppet/puppet.conf:

[agent]
  report = true
  report_server = $server
  report_port = $masterport

The preceding settings are enabled by default, and you could leave them out to achieve the normal submission behavior. You should only add them to the Puppet configuration file if you are changing them, such as to disable the sending of reports or to specify an alternate report processor.

Running Audit Inspections

If you have resources defined with the audit attribute, you can run the puppet inspect command on the node to generate a node report containing the detailed status of all audited resources. The node report will be submitted to the default Puppet server, or to the server passed on the command line:f

[vagrant@client ~]$ sudo puppet inspect
Notice: Finished inspection in 1.40 seconds

If you are using Puppet Dashboard, you can find the inspection report under the Node details page, as shown in “Reviewing a node’s history”.

If you have enabled a server-backed file bucket, you can add the --archive_files option to back up copies of audited file resources during the inspection:

$ sudo puppet inspect --archive_files --archive_file_server=puppet.example.com

The --archive_file_server argument is optional and defaults to the configured Puppet server.

If you make use of audit, you may want to run this from cron on all client nodes:

cron { 'audit-file-changes':
  command     => 'puppet inspect --archive_files',
  user        => 'root',
  hour        => '0',
  minute      => '0',
}

You can use Puppet Dashboard to review the report, or write a custom processor to analyze and compare reports. You’ll learn how to write a custom report processor in “Creating a Custom Report Processor”.

Storing Node Reports

The default behavior of a Puppet server is to store the reports in YAML format in the directory specified by the reportdir configuration parameter. With the settings we recommend in this book, this would default to the following settings in /etc/puppetlabs/puppet/puppet.conf:

[master]
  reports = store
  reportdir = /var/opt/puppetlabs/puppetserver/reports

You can examine this directory now and see the report from the client instance. As the report directories are owned and readable only by the Puppet server, you will have to either chown() the directories or become the puppet user to read the files:

[vagrant@puppetserver ~]$ sudo -u puppet bash
bash-4.2$ cd /var/opt/puppetlabs/puppetserver/reports/
bash-4.2$ ls -la
total 0
drwxr-x---  3 puppet puppet  31 Aug 16 21:23 .
drwxrwxr-x 12 puppet puppet 139 Aug  7 06:33 ..
drwxr-x---  2 puppet puppet  30 Aug 16 21:23 client.example.com

bash-4.2$ ls -la client.example.com
total 8
drwxr-x--- 2 puppet puppet   30 Aug 16 21:23 .
drwxr-x--- 3 puppet puppet   31 Aug 16 21:23 ..
-rw-r----- 1 puppet puppet 6706 Aug 16 21:23 201508162123.yaml

bash-4.2$ head -10 client.example.com/201508162123.yaml
--- !ruby/object:Puppet::Transaction::Report
metrics:
  resources: !ruby/object:Puppet::Util::Metric
    label: Resources
    name: resources
    values:
    - - total
      - Total
      - 7
    - - skipped

The report contains a log message for every action taken by the node, in addition to metrics about the number of resources processed, skipped, applied, or failed and the elapsed time Puppet spent evaluating and applying resources.

On the client node, you can find the same report file at /opt/puppetlabs/puppet/cache/state/last_run_report.yaml.

Logging Node Reports

If you are happy with log messages on the agent nodes, or their ability to send the logs to a centralized log server using the local logger, then you don’t need the log report handler. The Puppet agent will send each message at or above the configured log level to the configured log target (the default is syslog).

If your Puppet server has access to a centralized log server unavailable to the agent nodes, or the server is functioning as a log aggregator, then it can be helpful to have the server send all agent messages to its own logs. You can enable this by adding the log action to the list of report processors in /etc/puppetlabs/puppet/puppet.conf:

[master]
  reports = log,store

Transmitting Node Reports via HTTP

A Puppet server can be configured to submit node reports to another system. This facility is used by Puppet Dashboard and some alternative web dashboards for Puppet.

Enable transmission of node reports to a URL by adding the http action to the list of report processors, and specifying a target with with the reporturl setting in /etc/puppetlabs/puppet/puppet.conf:

[master]
  reports = http,store
  reporturl = https://puppet-dashboard.example.com/

We’ll cover installing and using Puppet Dashboard in Part IV.

You can create your own service to process reports delivered via HTTP. The data will arrive by POST method with a Content-Type of application/x-yaml. The YAML content will be the same as the contents of the file stored on disk on both the node and the master, as described earlier.

Transmitting Node Reports to PuppetDB

PuppetDB provides a report processor that can be used to submit node reports to the PuppetDB server. Enable transmission of node reports to PuppetDB by adding the puppetdb action to the list of report processors in /etc/puppetlabs/puppet/puppet.conf. The normal configuration for PuppetDB would look something like this:

[master]
  storeconfigs = true
  storeconfigs_backend = puppetdb
  reports = puppetdb, store

This will transmit the reports to the PuppetDB server specified in the /etc/puppetlabs/puppet/puppetdb.conf. This is covered briefly in Part IV.

Emailing Node Reports

Previous versions of Puppet included a report processor called tagmail, which would send email messages containing log messages that matched defined tags. This was removed from the Puppet core, and is now a separate Puppet module. To use this report processor, you’ll need to do the following on the Puppet server:

[vagrant@client ~]$ puppet module install puppetlabs-tagmail
Notice: Preparing to install into 
  /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
└── puppetlabs-tagmail (v2.1.0)

After installing it on the Puppet server, create a /etc/puppetlabs/puppet/tagmail.conf file containing a map of Puppet tags and recipients.

Each line of the [tagmap] section should contain one of the following formats:

• A list of tags or negated tags, followed by a colon and a comma-separated list of email addresses
• A list of log levels or negated log levels, followed by a colon and a comma-separated list of email addresses

Each log entry for a resource tagged with one of these tags, or matching the log level specified, will be added to an email sent to listed recipients. Here’s an example file showing you the ideas:

[tagmap]
# Log levels
all: log-archive@example.com
emerg: puppet-admins@example.com

# Tags
frontend, java: fe-dev-team@example.com, java-platform@example.com
database, mysql: dba@example.com
mongodb, !database: nosql-admins@example.com

[transport]
reportfrom = puppetserver@example.com
sendmail = /usr/sbin/sendmail

# Alternative direct SMTP delivery
#smtpserver = smtp-relay.example.com
#smtphelo = puppet.example.com
#smtpport = 25

The settings in the [transport] section of the file should be obvious to any experienced system admin. You can enable SMTP auth settings in your mailer of choice, or utilize basic SMTP delivery by enabling the smtpserver option.

Enable transmission of email reports by adding the tagmail action to the list of report processors in /etc/puppetlabs/puppet/puppet.conf:

[master]
  reports = store, tagmail

More documentation for tagmail can be found at puppetlabs/tagmail on the Puppet Forge.

Creating a Custom Report Processor

Custom report processors must be written in the Ruby language and follow the rules listed here. You can write a wrapper in Ruby that calls another program, as long as it works in the following manner:

• The Ruby script must be named according to the rules for Ruby symbols (alphanumeric only) with a trailing .rb suffix.
• It must have require 'puppet' at the top of the script.
• It must call Puppet::Reports.register_report() with a block of code, passing in its own name as a symbol as the only parameter.
• The block of code must contain:
  • A call to desc with a text or Markdown-formatted string that describes the report processor.
  • A defined method named process that does whatever you want it to do.

The process method will be an object of type Puppet::Transaction::Report. It can retrieve details of the report by querying the attributes shown at “Formats: Reports” on the Puppet docs site, or get YAML data by calling the self.to_yaml() method.

Here is an example report processor that creates a different log message for each Puppet agent run. It’s not unique or helpful as it stands, but it can be used as a starting point:

require 'puppet'
require 'syslog/logger'
log = Syslog::Logger.new 'logActionTime'

Puppet::Reports.register_report( :logActionTime ) do
  desc "Sends one log message containing agent action times."
  def process
    source = "#{self.time} #{self.host} "
    action = "did #{self.kind} with result #{self.status} "
    version = "for config #{self.configuration_version} "
    environment = "in environment #{self.environment}"
    log.info source + action + version + environment
  end
end

For a real report processor, you’d want to explore the self.resource_statuses and self.metrics hashes, or the self.logs array.

Add this report processor to the lib/puppet/reports directory of a Puppet module. If you don’t have a more appropriate module in mind, add it to the Puppet module you created in Part II.

Enable the report processor by adding its name to the list of report processors in /etc/puppetlabs/puppet/puppet.conf:

[master]
  reports = store, puppetdb, logActionTime