Hiera

Hiera allows you to create a hierarchy of node information. Using hiera, you can separate your variables and data from your modules. You start by defining what that hierarchy will be by ordering lookups in the main configuration file, hiera.yaml. The hierarchy is based on facts. Any fact can be used, even your own custom facts may be used. The values of the facts are then used as values for the YAML files stored in a directory, usually called hieradata. More information on hiera may be found on the Puppet Labs website at http://docs.puppetlabs.com/hiera/1.

Tip

Facts are case sensitive in hiera and templates; this could be important when writing your hiera.yaml script.

Configuring hiera

Hiera only needs to be installed on your worker nodes. Using the Puppet Labs repo, the package to install is hiera; our installation pulled down hiera-1.3.0-1.el6.noarch. The command-line hiera tool looks for the hiera configuration file, hiera.yaml, in /etc/hiera.yaml. Puppet will by default look for hiera.yaml in /etc/puppet/hiera.yaml. To use the command-line utility consistently with Puppet, symlink one to the other. I suggest making /etc/puppet/hiera.yaml the main file and /etc/hiera.yaml the link.

Tip

If you wish to use the /etc/hiera.yaml file, you can also specify hiera_config=/etc/hiera.yaml in /etc/puppet.conf.

The hieradata directory should also be under the /etc/puppet directory. We will create a directory to hold hieradata at /etc/puppet/hieradata and make the symlink between the hiera.yaml configuration files, as shown in the following commands:

worker1# mkdir /etc/puppet/hieradata
worker1# rm /etc/hiera.yaml
worker1# ln -s /etc/puppet/hiera.yaml /etc/hiera.yaml

Now we can create a simple hiera.yaml in /etc/puppet/hiera.yaml to show how the hierarchy is applied to a node, as shown in the following code snippet:

---
:hierarchy:
- "hosts/%{::hostname}"
- "roles/%{::role}"
- "%{::kernel}/%{::osfamily}/%{::lsbmajdistrelease} "
- "is_virtual/%{::is_virtual} "
- common
:backends:
- yaml
:yaml:
:datadir: '/etc/puppet/hieradata'

The lsbmajdistrelease fact requires that the Linux System Base (LSB) package be installed (redhat-lsb).

This hierarchy is quite basic. Hiera will look for a variable starting with the hostname of the node in the host's directory and then move to the top scope variable role in the directory roles. If a value is not found in roles, it will look in the directory /etc/puppet/hieradata/kernel/osfamily/ for a file named lsbmajdistrelease.yaml. On my test node, this would be /etc/puppet/hieradata/Linux/RedHat/6.yaml. If the value is not found there, then hiera will continue to look in hieradata/is_virtual/true.yaml (as my node is a virtual machine, the value of is_virtual will be true). If the value is still not found, the default file common.yaml will be tried. If the value is not found in common, then the command-line utility will return nil.

Tip

When using hiera in manifests, always set a default value, as failure to find anything in hiera will lead to a failed catalog (although having the node fail when this happens is also an often employed tactic).

As an example, we will set a variable syslogpkg to indicate which syslog package is used on our nodes. For EL6 machines, the package is rsyslog; for EL5, the package is syslog. Create two YAML files, one for EL6 at /etc/puppet/hieradata/Linux/RedHat/6.yaml using the following code:

---
syslogpkg: rsyslog  

Create another YAML file for EL5 at /etc/puppet/hieradata/Linux/RedHat/5.yaml using the following code:

---
syslogpkg: syslog

With these files in place, we can test our hiera by setting top scope variables (facts) from the command line. We run hiera three times, changing the value of lsbmajdistrelease each time, as shown in the following commands:

worker1# hiera syslogpkg ::kernel=Linux ::osfamily=RedHat ::lsbmajdistrelease=6
rsyslog
worker1# hiera syslogpkg ::kernel=Linux ::osfamily=RedHat ::lsbmajdistrelease=5
syslog
worker1# hiera syslogpkg ::kernel=Linux ::osfamily=RedHat ::lsbmajdistrelease=4
nil

In the previous commands, we change the value of lsbmajdistrelease from 6 to 5 to 4 to simulate the nodes running on EL6, EL5, and EL4. We do not have a 4.yaml file, so there is no setting of syslogpkg and hiera that returns nil.

Now to use hiera in our manifests, we can use the hiera function inline or set a variable using hiera. When using hiera, the syntax is hiera('variable','default'). The variable value is the key you are interested in looking at; the default value is the value to use when nothing is found in the hierarchy. Create a syslog module in /etc/puppet/modules/syslog/manifest/init.pp that starts syslog and makes sure the correct syslog is installed, as shown in the following code:

class syslog {
  $syslogpkg = hiera('syslogpkg','syslog')
  package {"$syslogpkg":
    ensure => 'installed',
  }
  service {"$syslogpkg":
    ensure => true,
    enable => true,
  }
}

Then create an empty /etc/puppet/manifests/site.pp file that includes syslog, as shown in the following code:

node default {
  include syslog
}

In this code, we set our default node to include the syslog module, and then we define the syslog module. The syslog module looks for the hiera variable syslogpkg to know which syslog package to install. Running this on our test node, we see that rsyslog is started as we are running EL6, as shown in the following commands:

node1# puppet agent -t
Info: Retrieving plugin
Info: Caching catalog for node1
Info: Applying configuration version '1388785169'
Notice: /Stage[main]/Syslog/Service[rsyslog]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Syslog/Service[rsyslog]: Unscheduling refresh on Service[rsyslog]
Notice: Finished catalog run in 0.71 seconds

Tip

If you didn't already disable the LDAP ENC we configured in the previous section, instructions are provided at the end of the LDAP backend section from this chapter.

In the enterprise, you want a way to automatically apply classes to nodes based on facts. This is part of a larger issue of separating the code of your modules from the data used to apply them. We will examine this issue in greater depth in Chapter 9, Roles and Profiles. Hiera has a function that makes this very easy—hiera_include. Using hiera_include you can have hiera apply classes to a node based upon the hierarchy.

Using hiera_include

To use hiera_include, we set a hiera variable to hold the name of the classes we would like applied to the nodes. By convention, this is called classes, but it could be anything. We'll also set a variable role that we'll use in our new base class. We modify site.pp to include all classes defined in the hiera variable classes. We also set a default value should no values be found; this way we guarantee that catalogs will compile and that all nodes receive at least the base class. Edit /etc/puppet/manifest/site.pp as follows:

node default {
  hiera_include('classes', 'base')
}

For the base class, we'll just set the motd file as we've done previously. We'll also set a welcome string in hiera; in common.yaml, we'll set this to something generic and override the value in a hostname-specific YAML file. Edit the base class in /etc/puppet/modules/base/manifests/init.pp as follows:

class base {
  $welcome = hiera('welcome','Welcome')
  file {'/etc/motd':
    mode => '0644',
    owner => '0',
    group => '0',
    content => inline_template("<%= welcome %>\nManaged Node: <%= hostname
%>\nManaged by Puppet version <%= puppetversion %>\n"),
  }
}

This is our base class; it uses an inline template to set up the "message of the day" file (/etc/motd). We then need to set the welcome information in hieradata; edit /etc/puppet/hieradata/common.yaml to include the default welcome message, as shown in the following code snippet:

---
welcome: 'Welcome to Example.com'
classes: - 'base'
syslog: 'nothing'

Now we can run Puppet on our node1 machine; after the successful run, our /etc/motd has the following contents:

Welcome to Example.com
Managed Node: node1
Managed by Puppet version 3.4.1

Now to test if our hierarchy is working as expected, we'll create a YAML file specifically for node1, /etc/puppet/hieradata/hosts/node1.yaml as follows:

---
welcome: 'Welcome to our default node'

Again, we run Puppet on node1 and examine the contents of /etc/motd, as shown in the following code:

Welcome to our default node
Managed Node: node1
Managed by Puppet version 3.4.1

Now that we have verified that our hierarchy performs as we expect, we can use hiera to apply a class to all nodes based on a fact. In this example we'll use the is_virtual fact to do some performance tuning on our virtual machines. We'll create a virtual class in /etc/puppet/modules/virtual/manifests/init.pp, which installs the tuned package. It then sets the tuned profile to virtual-guest and starts the tuned service, as shown in the following code:

class virtual {
  # performance tuning for virtual machine
  package {'tuned':
    ensure => 'present',
  }
  service {'tuned':
    enable => true,
    ensure => true,
    require => Package['tuned']
  }
  exec {'set tuned profile':
    command => '/usr/sbin/tuned-adm profile virtual-guest',
    unless => '/bin/grep -q virtual-guest /etc/tune-profiles/activeprofile',
  }
}

Tip

In a real-world example, we'd verify that we only apply this to nodes running on EL6.

This module ensures that the tuned package is installed and the tuned service is started. It then verifies that the current tuned profile is set to virtual-guest (using a grep statement in the unless parameter to the exec), if the current profile is not virtual-guest, the profile is changed to virtual-guest using tuned-adm.

Tip

Tuned is a tuning daemon included on enterprise Linux systems, which configures several kernel parameters related to scheduling and I/O operations.

To ensure that this class is applied to all virtual machines, we simply need to add it to the classes hiera variable in /etc/puppet/hieradata/is_virtual/true.yaml, as shown in the following snippet:

---
classes: - 'virtual'

Now our test node node1 is indeed virtual, so if we run Puppet now, the virtual class will be applied to the node, and we will see that the tuned profile is set to virtual-guest. Running tuned-admin active on the host returns the currently active profile; when we run it initially the command is not available as the tuned rpm has not been installed yet, as you can see in the following commands:

node1# tuned-adm active
-bash: tuned-adm: command not found
node1# puppet agent -t
Info: Retrieving plugin
Info: Caching catalog for node1
Info: Applying configuration version '1388817444'
Notice: /Stage[main]/Virtual/Package[tuned]/ensure: created
Notice: /Stage[main]/Virtual/Exec[set tuned profile]/returns: executed successfully
Notice: Finished catalog run in 9.65 seconds
node1# tuned-adm active
Current active profile: virtual-guest
Service tuned: enabled, running
Service ktune: enabled, running

This example shows the power of using hiera with hiera_include and a well-organized hierarchy. Using this method, we can have classes applied to nodes based on facts and reduce the need for custom classes on nodes. We do, however, have the option of adding classes per node since we have a hosts/%{::hostname} entry in our hierarchy. If you had, for instance, a module that only needed to be installed on 32-bit systems, you could make an entry in hiera.yaml for %{::architecture} and only create an i686.yaml file that contained the class in question. Building up your classes in this fashion reduces the complexity of your individual node configurations.

Another great feature of hiera is its ability to automatically fill in values for parameterized class attributes. For this example, we will create a class called resolver and set the search parameter for our /etc/resolv.conf file using augeas.

Note

Augeas is a tool for modifying configuration files as though they were objects. For more information on augeas, visit the project website at http://augeas.net. In this example, we will use augeas to modify only a section of the /etc/resolv.conf file.

First, we will create a resolver class as follows in /etc/puppet/modules/resolver/manifests/init.pp:

class resolver($search = "example.com") {
  augeas { 'set resolv.conf search':
    context => '/files/etc/resolv.conf',
    changes => [
      "set search/domain '${search}'"
    ],
  }
}

Then we add resolver to our classes in /etc/puppet/hieradata/hosts/node1.yaml so as to have resolver applied to our node, as shown in the following code:

---
welcome: 'Welcome to our default node'
classes: - resolver

Now we run Puppet on node1 as shown in the following commands; augeas will change the resolv.conf file to have the search domain set to the default example.com.

node1# puppet agent -t
Info: Retrieving plugin
Info: Caching catalog for node1
Info: Applying configuration version '1388818864'
Notice: Augeas[set resolv.conf search](provider=augeas):
--- /etc/resolv.conf 2014-01-04 01:59:43.769423982 -0500
+++ /etc/resolv.conf.augnew 2014-01-04 02:00:09.552425174 -0500
@@ -1,2 +1,3 @@
; generated by /sbin/dhclient-script
nameserver 192.168.122.1
+search example.com

Notice: /Stage[main]/Resolver/Augeas[set resolv.conf search]/returns: executed successfully
Notice: Finished catalog run in 1.09 seconds

Now, to get hiera to override the default parameter for the parameterized class resolver, we simply set the hiera variable resolver::search in our /etc/puppet/hieradata/hosts/node1.yaml file, as shown in the following code:

---
welcome: 'Welcome to our default node'
classes: - resolver
resolver::search: 'devel.example.com'

Running puppet agent another time on node1 will change the search from example.com to devel.example.com using the value from the hiera hierarchy file, as you can see in the following commands:

[root@node1 ~]# puppet agent -t
Info: Retrieving plugin
Info: Caching catalog for node1.example.com
Info: Applying configuration version '1388818864'
Notice: Augeas[set resolv.conf search](provider=augeas):
--- /etc/resolv.conf 2014-01-04 02:09:00.192424927 -0500
+++ /etc/resolv.conf.augnew 2014-01-04 02:13:24.815425173 -0500
@@ -1,4 +1,4 @@
; generated by /sbin/dhclient-script
nameserver 192.168.122.1
domain example.com
-search example.com
+search devel.example.com

Notice: /Stage[main]/Resolver/Augeas[set resolv.conf search]/returns: executed successfully
Notice: Finished catalog run in 1.07 seconds

By building up your catalog in this fashion, it's possible to override parameters to any class. At this point, our node1 machine has the virtual, resolver and base classes, but our site manifest (/etc/puppet/manifests/site.pp) only has a hiera_include line, as shown in the following code:

node default {
  hiera_include('classes',base)
}

In the enterprise, this means that you can add new hosts without modifying your site manifest and that you can customize the classes and any parameters to those classes.

Two other functions exist for using hiera; they are hiera_array and hiera_hash. These functions do not stop at the first match found in hiera and instead return either an array or hash of all the matches. This can also be used in powerful ways to build up definitions of variables. One good use of this is in setting the nameservers a node will query. Using hiera_array instead of hiera function, you can not only set nameservers based on the hostname of the node or some other facts, but also have the default name servers from your common.yaml file applied to the node.