Hiera

Hiera allows you to create a hierarchy of node information. Using Hiera, you can separate your variables and data from your modules. You start by defining what that hierarchy will be, by ordering lookups in the main configuration file, hiera.yaml. The hierarchy is based on facts. Any fact can be used, even your own custom facts may be used. The values of the facts are then used as values for the YAML files stored in a directory, usually called hieradata. More information on Hiera can be found on the Puppet Labs website at http://docs.puppetlabs.com/hiera/latest.

Tip

Facts are case sensitive in Hiera and templates. This could be important when writing your hiera.yaml script.

Configuring Hiera

Hiera only needs to be installed on your Puppet master nodes. Using the Puppet Labs repo, Hiera is installed by the puppet-agent package. Our installation pulled down puppet-agent-1.2.2-1.el7.x86_64, which installs Hiera version 3.0.1, as shown here:

[thomas@stand ~]$ hiera --version
3.0.1

Previous versions of the command-line Hiera tool would expect the Hiera configuration file, hiera.yaml, in /etc/hiera.yaml. The previous versions of Puppet would expect the configuration file in /etc/puppet/hiera.yaml or /etc/puppetlabs/puppet/hiera.yaml, depending on the version of Puppet. This caused some confusion, as the command-line utility will, by default, search in a different file than Puppet. This problem has been corrected in Puppet 4; the Hiera configuration file is now located in the /etc/puppetlabs/code directory. The default location for the hieradata directory includes the value of the current environment and is located at /etc/puppetlabs/code/environments/%{environment}/hieradata.

Now, we can create a simple hiera.yaml in /etc/puppet/hiera.yaml to show how the hierarchy is applied to a node, as shown here:

---
:hierarchy:
  - "hosts/%{::hostname}"
  - "roles/%{::role}"
  - "%{::kernel}/%{::os.family}/%{:os.release.major}"
  - "is_virtual/%{::is_virtual}"
  - common
:backends:
  - yaml
:yaml:
  :datadir:

This hierarchy is quite basic. Hiera will look for a variable starting with the hostname of the node in the host's directory and then move to the top scope variable role in the directory roles. If a value is not found in the roles, it will look in the /etc/puppet/hieradata/kernel/osfamily/ directory (where kernel and osfamily will be replaced with the Facter values for these two facts) for a file named lsbmajdistrelease.yaml. On my test node, this would be /etc/puppet/hieradata/Linux/RedHat/6.yaml. If the value is not found there, then Hiera will continue to look in hieradata/is_virtual/true.yaml (as my node is a virtual machine, the value of is_virtual will be true). If the value is still not found, then the default file common.yaml will be tried. If the value is not found in common, then the command-line utility will return nil.

When using the hiera function in manifests, always set a default value, as failure to find anything in Hiera will lead to a failed catalog (although having the node fail when this happens is often used intentionally as an indicator of a problem).

As an example, we will set a variable syslogpkg to indicate which syslog package is used on our nodes. The syslog package is responsible for system logging. For EL7 and EL6 machines, the package is rsyslog; for EL5, the package is syslog. Create three YAML files, one for EL6 and EL7 at /etc/puppetlabs/code/environments/production/hieradata/Linux/RedHat/7.yaml using the following code:

---
syslogpkg: rsyslog  

Create another YAML file for EL5 at /etc/puppetlabs/code/environments/production/hieradata/Linux/RedHat/5.yaml using the following code:

---
syslogpkg: syslog

With these files in place, we can test our Hiera by setting top scope variables (facts), using a YAML file. Create a facts.yaml file with the following content:

is_virtual: true
kernel: Linux
os:
  family: RedHat
  release:
    major: "7"

We run Hiera three times, changing the value of os.release.major from 7 to 5 to 4 and observe the following results:

[thomas@stand ~]$ hiera syslogpkg -y facts.yaml environment=production
rsyslog
[thomas@stand ~]$ hiera syslogpkg -y facts.yaml environment=production
syslog
[thomas@stand ~]$ hiera syslogpkg -y facts.yaml environment=production
nil

In the previous commands, we change the value of os.release.major from 7 to 5 to 4 to simulate the nodes running on EL7, EL5 and EL4. We do not have a 4.yaml file, so there is no setting of syslogpkg and hiera that returns nil.

Now to use Hiera in our manifests, we can use the hiera function inline or set a variable using Hiera. When using Hiera, the syntax is hiera('variable','default'). The variable value is the key you are interested in looking at; the default value is the value to use when nothing is found in the hierarchy. Create a syslog module in /etc/puppet/modules/syslog/manifest/init.pp that starts syslog and makes sure the correct syslog is installed, as shown here:

class syslog {
  $syslogpkg = hiera('syslogpkg','syslog')
  package {"$syslogpkg":
    ensure => 'installed',
  }
  service {"$syslogpkg":
    ensure => true,
    enable => true,
  }
}

Then create an empty /etc/puppet/manifests/site.pp file that includes syslog, as shown here:

node default {
  include syslog
}

In this code, we set our default node to include the syslog module and then we define the syslog module. The syslog module looks for the Hiera variable syslogpkg to know which syslog package to install. Running this on our client node, we see that rsyslog is started as we are running EL7, as shown here:

[thomas@client ~]$ sudo puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for client.example.com
Info: Applying configuration version '1442381098'
Notice: /Stage[main]/Syslog/Package[rsyslog]/ensure: created
Notice: /Stage[main]/Syslog/Service[rsyslog]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Syslog/Service[rsyslog]: Unscheduling refresh on Service[rsyslog]
Notice: Applied catalog in 7.83 seconds

Note

If you haven't already disable the LDAP ENC, which we configured in the previous section, the instructions are provided at the end of the LDAP backend section of this chapter.

In the enterprise, you want a way to automatically apply classes to nodes based on facts. This is part of a larger issue of separating the code of your modules from the data used to apply them. We will examine this issue in greater depth in Chapter 9, Roles and Profiles. Hiera has a function that makes this very easy—hiera_include. Using hiera_include, you can have Hiera apply classes to a node based upon the hierarchy.

Using hiera_include

To use hiera_include, we set a Hiera variable to hold the name of the classes we would like to apply to the nodes. By convention, this is called classes, but it could be anything. We'll also set a variable role that we'll use in our new base class. We modify site.pp to include all the classes defined in the Hiera variable classes. We also set a default value if no values are found; this way we can guarantee that the catalogs will compile and all the nodes receive at least the base class. Edit /etc/puppetlabs/code/environments/production/manifest/site.pp, as follows:

node default {
  hiera_include('classes', 'base')
}

For the base class, we'll just set the motd file, as we've done previously. We'll also set a welcome string in Hiera. In common.yaml, we'll set this to something generic and override the value in a hostname-specific YAML file. Edit the base class in /etc/puppetlabs/code/environments/production/modules/base/manifests/init.pp, as follows:

class base {
  $welcome = hiera('welcome','Welcome')
  file {'/etc/motd':
    mode => '0644',
    owner => '0',
    group => '0',
    content => inline_template("<%= @welcome %>\nManaged Node: <%= @hostname %>\nManaged by Puppet version <%= @puppetversion %>\n"),
  }
}

This is our base class; it uses an inline template to set up the message of the day file (/etc/motd). We then need to set the welcome information in hieradata; edit /etc/puppet/hieradata/common.yaml to include the default welcome message, as shown here:

---
welcome: 'Welcome to Example.com'
classes:
  - 'base'
syslogpkg: 'nothing'

Now we can run Puppet on our node1 machine. After the successful run, our /etc/motd file has the following content:

Welcome to Example.com
Managed Node: client
Managed by Puppet version 4.2.1

Now, to test if our hierarchy is working as expected, we'll create a YAML file specifically for client, /etc/puppetlabs/code/environments/production/hieradata/hosts/client.yaml, as follows:

---
welcome: 'Welcome to our default node'

Again, we run Puppet on client and examine the contents of /etc/motd, as shown here:

[thomas@client ~]$ cat /etc/motd
Welcome to our default node
Managed Node: client
Managed by Puppet version 4.2.1

Now that we have verified that our hierarchy performs as we expect, we can use Hiera to apply a class to all the nodes based on a fact. In this example, we'll use the is_virtual fact to do some performance tuning on our virtual machines. We'll create a virtual class in /etc/puppet/modules/virtual/manifests/init.pp, which installs the tuned package. It then sets the tuned profile to virtual-guest and starts the tuned service, as shown here:

class virtual {
  # performance tuning for virtual machine
  package {'tuned':
    ensure => 'present',
  }
  service {'tuned':
    enable => true,
    ensure => true,
    require => Package['tuned']
  }
  exec {'set tuned profile':
    command => '/usr/sbin/tuned-adm profile virtual-guest',
    unless => '/bin/grep -q virtual-guest /etc/tune-profiles/activeprofile',
  }
}

Note

In a real-world example, we'd verify that we only apply this to nodes running on EL6 or EL7.

This module ensures that the tuned package is installed and the tuned service is started. It then verifies that the current tuned profile is set to virtual-guest (using a grep statement in the unless parameter to the exec). If the current profile is not virtual-guest, the profile is changed to virtual-guest using tuned-adm.

Note

Tuned is a tuning daemon included on enterprise Linux systems, which configures several kernel parameters related to scheduling and I/O operations.

To ensure that this class is applied to all virtual machines, we simply need to add it to the classes Hiera variable in /etc/puppet/hieradata/is_virtual/true.yaml, as shown here:

---
classes:
  - 'virtual'

Now our test node client is indeed virtual, so if we run Puppet now, the virtual class will be applied to the node and we will see that the tuned profile is set to virtual-guest. Running tuned-adm active on the host returns the currently active profile. When we run it initially, the command is not available as the tuned rpm has not been installed yet, as you can see here:

[thomas@client ~]$ sudo tuned-adm active
sudo: tuned-adm: command not found

Next, we run puppet agent to set the active profile (tuned is installed by default on EL7 systems):

[thomas@client ~]$ sudo puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for client.example.com
Info: Applying configuration version '1442383469'
Notice: /Stage[main]/Virtual/Exec[set tuned profile]/returns: executed successfully
Notice: Applied catalog in 1.15 seconds

Now, when we run tuned-adm active we see that the active profile has been changed accordingly:

[thomas@client ~]$ sudo tuned-adm active
Current active profile: virtual-guest

This example shows the power of using Hiera, with hiera_include and a well-organized hierarchy. Using this method, we can have classes applied to nodes based on facts and reduce the need for custom classes on nodes. We do, however, have the option of adding classes per node since we have a hosts/%{hostname} entry in our hierarchy. If you had, for instance, a module that only needed to be installed on 32-bit systems, you could make an entry in hiera.yaml for %{architecture} and only create an i686.yaml file that contained the class in question. Building up your classes in this fashion reduces the complexity of your inpidual node configurations.

In fact, in Puppet version 3, architecture is available as both architecture and os.architecture.

Another great feature of Hiera is its ability to automatically fill in values for parameterized class attributes. For this example, we will create a class called resolver and set the search parameter for our /etc/resolv.conf file using Augeas.

Note

Augeas is a tool to modify configuration files as though they were objects. For more information on Augeas, visit the project website at http://augeas.net. In this example, we will use Augeas to modify only a section of the /etc/resolv.conf file.

First, we will create a resolver class as follows in /etc/puppetlabs/code/environments/production/modules/resolver/manifests/init.pp:

class resolver($search = "example.com") {
  augeas { 'set resolv.conf search':
    context => '/files/etc/resolv.conf',
    changes => [
      "set search/domain '${search}'"
    ],
  }
}

Then we add resolver to our classes in /etc/puppetlabs/code/environments/production/hieradata/hosts/client.yaml, so as to have resolver applied to our node, as shown here:

---
welcome: 'Welcome to our default node'
classes:
  - resolver

Now we run Puppet on the client; Augeas will change the resolv.conf file to have the search domain set to the default example.com:

[thomas@client ~]$ sudo puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for client.example.com
Info: Applying configuration version '1442411609'
Notice: Augeas[set resolv.conf search](provider=augeas): 
--- /etc/resolv.conf 2015-09-16 09:53:15.727804673 -0400
+++ /etc/resolv.conf.augnew 2015-09-16 09:53:28.108995714 -0400
@@ -2,3 +2,4 @@
 nameserver 8.8.8.8
 nameserver 8.8.4.4
 domain example.com
+search example.com

Notice: /Stage[main]/Resolver/Augeas[set resolv.conf search]/returns: executed successfully
Notice: Applied catalog in 1.41 seconds

Now, to get Hiera to override the default parameter for the parameterized class resolver, we simply set the Hiera variable resolver::search in our /etc/puppetlabs/code/environments/production/hieradata/hosts/client.yaml file, as shown here:

---
welcome: 'Welcome to our default node'
classes:
  - resolver
resolver::search: 'devel.example.com'

Running puppet agent another time on node1 will change the search from example.com to devel.example.com, using the value from the Hiera hierarchy file, as you can see here:

[thomas@client ~]$ sudo puppet agent -t
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for client.example.com
Info: Applying configuration version '1442411732'
Notice: Augeas[set resolv.conf search](provider=augeas): 
--- /etc/resolv.conf 2015-09-16 09:53:28.155509013 -0400
+++ /etc/resolv.conf.augnew 2015-09-16 09:55:30.656393427 -0400
@@ -2,4 +2,4 @@
 nameserver 8.8.8.8
 nameserver 8.8.4.4
 domain example.com
-search example.com
+search devel.example.com

Notice: /Stage[main]/Resolver/Augeas[set resolv.conf search]/returns: executed successfully
Notice: Applied catalog in 0.94 seconds

By building up your catalog in this fashion, it's possible to override parameters of any class. At this point, our client machine has the virtual, resolver and base classes, but our site manifest (/etc/puppet/manifests/site.pp) only has a hiera_include line, as shown here:

node default {
  hiera_include('classes',base)
}

In the enterprise, this means that you can add new hosts without modifying your site manifest and that you can customize the classes and any parameters to those classes.

Note

Using hiera_include to specify the classes assigned to a node ensures that the node cannot assign itself to a different role. Some site.pp manifests will use a fact to determine the classes to be applied, this will allow anyone who can control facts on the node to change the classes on the node and potentially access configurations for different types of nodes.

Two other functions exist for using Hiera; they are hiera_array and hiera_hash. These functions do not stop at the first match found in Hiera and instead return either an array or hash of all the matches. This can also be used in powerful ways to build up definitions of variables. One good use of this is in setting the name servers a node will query. Using hiera_array instead of the hiera function, you can not only set nameservers based on the hostname of the node or some other facts, but also have the default nameservers from your common.yaml file applied to the node.