Organizing the nodes with ENC

An ENC is a script that is run on the Puppet master, or the host compiling the catalog, to determine which classes are applied to the node. The ENC script can be written in any language, and it receives as a command-line argument certname (certificate name) from the node. In most cases, this will be the Fully Qualified Domain Name (FQDN) of the node; we will assume that the certname setting has not been explicitly set and that the FQDN of our nodes is being used.

Tip

We will only use the hostname portion as the FQDN can be unreliable in some instances. Across your enterprise, the naming convention of the host should not allow for multiple machines to have the same hostname. The FQDN is determined by a fact; this fact is the union of the hostname fact and the domain fact. The domain fact on Linux is determined by running the command hostname -f. If DNS is not configured correctly or reverse records do not exist, the domain fact will not be set and the FQDN will also not be set, as shown in the following commands:

# facter domain
example.com
# facter fqdn
node1.example.com
# mv /etc/resolv.conf /etc/resolv.conf.bak
# facter domain
# facter fqdn
#

The output of the ENC script is a YAML file, which defines the classes, variables, and environment for the node.

Tip

Unlike site.pp, the ENC script can only assign classes, make top-scope variables, and set the environment of the node. Environment is only set from ENC on version 3 and above of Puppet.

A simple example

To use ENC, we need to make one small change in our Puppet worker machine. We'll add the node_terminus and external_nodes lines to the [master] section of puppet.conf, as shown in the following code (we only need make this change on the worker machines as this is concerned with catalog compilation only):

[master]
    node_terminus = exec
    external_nodes = /usr/local/bin/simple_node_classifier

Tip

The puppet.conf files need not be the same across our installation; workers and CA machines may have different settings.

Our first example, as shown in the following code snippet, will be written in Ruby and live in the file /usr/local/bin/simple_node_classifier:

#!/bin/env ruby
require 'yaml'

# create an empty hash to contain everything
@enc = Hash.new
@enc["classes"] = Hash.new
@enc["classes"]["base"] = Hash.new
@enc["parameters"] = Hash.new
@enc["environment"] = 'production'
#convert the hash to yaml and print
puts @enc.to_yaml
exit(0)

Make this script executable and test it on the command line as shown in the following snippet:

# chmod 755 /usr/local/bin/simple_node_classifier
# /usr/local/bin/simple_node_classifier
--- 
classes: 
 base: {}
environment: production
parameters: {}

This script returns a properly formatted YAML file.

Tip

YAML files start with three dashes (---); they use colons (:) to separate parameters from values, and hyphens (-) to separate multiple values (arrays). For more information on YAML, visit http://www.yaml.org/.

If you use a language such as Ruby or Python, you do not need to know the syntax of YAML as the built-in libraries take care of the formatting for you. The following is the same example in Python. To use the Python example, you will need to install PyYAML, that is, the Python YAML interpreter as shown in the following commands:

# yum install PyYAML
Installed:
 PyYAML.x86_64 0:3.10-3.el6

The Python version starts with an empty dictionary. We then use sub-dictionaries to hold the classes, parameters, and environment. We will call our Python example /usr/local/bin/simple_node_classifier_2. The following is our example:

#!/bin/env python
import yaml
import sys
# create an empty hash
enc = {}
enc["classes"] = {}
enc["classes"]["base"] = {}
enc["parameters"] = {}
enc["environment"] = 'production'
# output the ENC as yaml
print "---"
print yaml.dump(enc)
sys.exit(0)

Make /usr/local/bin/simple_node_classifier_2 executable and run it using the following commands:

worker1# chmod 755 /usr/local/bin/simple_node_classifier_2
worker1# /usr/local/bin/simple_node_classifier_2 
---
classes:
 base: {}
environment: production
parameters: {}

Tip

The order of the lines below the --- may be different on your machine; when Python dumps the hash of values, the order is not specified.

The Python script outputs the same YAML as the Ruby code. We will now define the base class referenced in our ENC script, as shown in the following code snippet:

class base {
  file {'/etc/motd':
    mode => '0644',
    owner => '0',
    group => '0',
    content => inline_template("Managed Node: <%= hostname %>\nManaged by Puppet version <%= puppetversion %>\n"),
  }
}

Now that our base class is defined, when we run Puppet on our node, we will see that our message of the day (/etc/motd) has been updated using an inline template, as shown in the following command-line output:

node1# puppet agent -t
Info: Retrieving plugin
Info: Caching catalog for node1
Info: Applying configuration version '1386748797'
Notice: /File[/etc/motd]/ensure: defined content as '{md5}ad29f471b2cbf5754c706cdc0a54684b'
Notice: Compiled on worker1
Notice: /Stage[main]//Node[default]/Notify[Compiled on worker1]/message: defined 'message' as 'Compiled on worker1'
Notice: This is an example
Notice: /Stage[main]/Example/Notify[This is an example]/message: defined 'message' as 'This is an example'
Notice: Finished catalog run in 0.11 seconds
node1# cat /etc/motd
Managed Node: node1
Managed by Puppet version 3.3.2

Since ENC is only given one piece of data ever, that is, FQDN (certname), we need to create a naming convention that provides us with enough information to determine which classes should be applied to the node.

Hostname strategy

In an enterprise, it's important that your hostnames are meaningful. By meaningful I mean that the hostname should give you as much information as possible about the machine; when encountering a machine in a large installation, it is very likely that you did not build the machine. You need to know as much as possible about the machine just from its name. The following key points should be readily determined from the hostname:

• Operating system
• Application/role
• Location
• Environment
• Instance

It is important that the convention is standardized and consistent. In our example, let us suppose that the application is the most important component for our organization, so we put that first, and the physical location comes next (which datacenter), followed by the operating system, environment, and instance number. The instance number will be used when you have more than one machine with the same role, location, environment, and operating system. Since we know that the instance number will always be a number, we can omit the underscore between the operating system and environment, making the hostname a little easier to type and remember.

Your enterprise may have more or less information, but the principle will remain the same. To delineate our components, we will use underscores (_); some companies rely on a fixed length for each component of the hostname so as to mark the individual components of the hostname by position alone.

In our example, we will have the following environments:

• p: This stands for production
• n: This stands for non-production
• d: This stands for development/testing/lab

Our applications will be of the following types:

• web
• db

Our operating system will be Linux, which we will shorten to just l, and our location will be our main datacenter (main). So, a production web server on Linux in the main datacenter would have the hostname web_main_lp01.

Tip

If you think you are going to have more than 99 instances of any single service, you might want to have another leading zero to the instance number (001).

Now this looks pretty good. We know that this is a web server in our main datacenter. It's running on Linux, and it's the first machine like this in production. Now that we have a nice convention like this, we need to modify our ENC to use the convention to glean all this information from the hostname.

Modified ENC using hostname strategy

We'll build our Python ENC script (/usr/local/bin/simple_node_classifier_2) and update it to use the new hostname strategy as shown in the following commands:

#!/bin/env python
# Python ENC
# receives fqdn as argument

import yaml
import sys
"""output_yaml renders the hash as yaml and exits cleanly"""
def output_yaml(enc):
 # output the ENC as yaml
 print "---"
 print yaml.dump(enc)
 quit()

Python is very particular about spacing; if you are new to Python, take care to copy the indenting exactly as given in the previous snippet.

We define a function to print the YAML and exit the script as shown in the following commands; if the hostname doesn't match our naming standards, we'll exit the script early:

# create an empty hash
enc = {}
enc["classes"] = {}
enc["classes"]["base"] = {}
enc["parameters"] = {}

try:
 hostname=sys.argv[1]
except:
 # need a hostname
 sys.exit(10)

Exit the script early if the hostname is not defined. This is a minimum requirement, we should never reach this point.

We split the hostname using underscores (_) in an array called parts; we then assign indexes of parts to role, location, os, environment, and instance, as shown in the following commands:

# split hostname on _
try:
 parts = hostname.split('_')
 role = parts[0]
 location = parts[1]
 os = parts[2][0]
 environment = parts[2][1]
 instance = parts[2][2:]

We are expecting hostnames to conform to the standard; if you cannot guarantee this, then you would have to use something like the regular expression module to deal with exceptions to the naming standard.

except:
 # hostname didn't conform to our standard
 # include a class which notifies us of the problem
 enc["classes"]["hostname_problem"] = hostname
 output_yaml(enc)
 raise SystemExit

We wrapped the previous assignments in a try statement; in this except statement, we exit printing the YAML and assign a class named hostname_problem. This class would be used to alert us in the console or reporting system that this host has a problem.

The environment is a single character in the hostname; hence, we use a dictionary to assign a full name to the environment, as shown in the following snippet:

# map environment from hostname into environment
environments = {}
environments['p'] = 'production'
environments['n'] = 'nonprod'
environments['d'] = 'devel'
environments['s'] = 'sbx'
try:
 enc["environment"] = environments[environment]
except:
 enc["environment"] = 'undef'

The following commands are used to map a role from hostname into role:

# map role from hostname into role
enc["classes"][role] = {}

Next, we assign top scope variables to the node based on the values we obtained from the parts array previously:

# set top scope variables
enc["parameters"]["enc_hostname"] = hostname
enc["parameters"]["role"] = role
enc["parameters"]["location"] = location
enc["parameters"]["os"] = os
enc["parameters"]["instance"] = instance

output_yaml(enc)

Heading back to web_main_lp01, we run Puppet, sign the certificate on our puppetca machine, and then run Puppet again to verify that the web class is applied, as shown in the following commands:

web_main_lp01# puppet agent -t
Info: Retrieving plugin
Info: Caching catalog for web-main-lp01
Info: Applying configuration version '1386834979'
Notice: /File[/etc/motd]/ensure: defined content as '{md5}a828f52c2447032b1864405626f4e3a4'
Notice: /Stage[main]/Web/Package[httpd]/ensure: created
Notice: /Stage[main]/Web/Service[httpd]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Web/Service[httpd]: Unscheduling refresh on Service[httpd]
Notice: Finished catalog run in 10.14 seconds

Our machine has been installed as a web server without any intervention on our part; the system knew which classes to apply to the machine based solely on the hostname. Now, if we try to run Puppet against our node1 machine created earlier, our ENC includes the class hostname_problem with the parameter of the hostname passed to it. We can create this class to capture the problem and notify us. Create the hostname_problem module in /etc/puppet/modules/hostname_problem/manifests/init.pp, as shown in the following snippet:

class hostname_problem ($enc_hostname) {
 notify {"WARNING: $enc_hostname ($::ipaddress) doesn't conform to naming standards": }
}

Now when we run Puppet on our node1 machine, we will get a useful warning that node1 isn't a good hostname for our enterprise, as you can see in the following commands:

node1# puppet agent -t
Info: Retrieving plugin
Info: Caching catalog for node1
Info: Applying configuration version '1386916930'
Notice: WARNING: node1 (192.168.122.132) doesn't conform to naming standards
Notice: /Stage[main]/Hostname_problem/Notify[WARNING: node1 (192.168.122.132) doesn't conform to naming standards]/message: defined 'message' as 'WARNING: node1 (192.168.122.132) doesn't conform to naming standards'
Notice: Finished catalog run in 0.05 seconds

Your ENC can be customized much further than this simple example; you have the power of Python, Ruby, or any other language you wish to use. You could connect to a database and run some queries to determine classes to install. For example, if you have a CMDB at your enterprise, you could connect to the CMDB and retrieve information based on the FQDN of the node and apply classes based on that information. You could connect to a URI and retrieve a catalog (dashboard and foreman do something similar). There are many ways to expand this concept. In the next section, we'll look at using LDAP to store class information.

LDAP backend

If you already have an LDAP implementation in which you can extend the schema, then you can use the LDAP node terminus that ships with Puppet. Using this schema adds a new objectclass called puppetclass. Using this objectclass, you can set the environment, set top scope variables, and include classes. The LDAP schema that ships with Puppet includes puppetClass, parentNode, environment, and the puppetVar attributes that are assigned to the objectclass named puppetClient. LDAP experts should note that all four of these attributes are marked as optional and the objectclass named puppetClient is non-structural. To use the LDAP terminus, you must have a working LDAP implementation, apply the Puppet schema to that installation and add the ruby-ldap package to your workers (to allow Puppet to query for node information).

OpenLDAP configuration

We'll begin by setting up a fresh OpenLDAP implementation and adding a puppet schema. Create a new machine and install openldap-servers; my installation installed the version openldap-servers-2.4.23-32.el6_4.1.x86_64. This version requires configuration with OLC (OpenLDAP Configuration or runtime configuration); further information on OLC can be obtained at http://www.openldap.org/doc/admin24/slapdconf2.html. OLC configures LDAP using LDAP.

After installing openldap-servers, your configuration will be in /etc/openldap/slapd.d/cn=config. There is a file named olcDatabase={2}.bdb.ldif in this directory. Edit the file and change the following lines:

olcSuffix: dc=example,dc=com
olcRootDN: cn=Manager,dc=example,dc=com
olcRootPW: packtpub

Tip

The olcRootPW line is not present in the default file, you will have to add it here. If you're going into production with LDAP, you should set olcDbConfig parameters as outlined at http://www.openldap.org/doc/admin24/slapdconf2.html.

These lines set the top-level location for your LDAP and the password for RootDN. This password is in plain text; a production installation would use SSHA encryption. You will be making schema changes, so you must also edit olcDatabase={0}config.ldif and set rootDN and rootPW. For our example, we will use the default rootDN value and set the password to packtpub, as shown in the following commands:

olcRootDN: cn=config
olcRootPW: packtpub

Tip

You would want to keep this RootDN value and the previous RootDN values separate so that this RootDN value is the only one that can modify schema and top-level configuration parameters.

Next, use ldapsearch (provided by the openldap-clients package, which has to be installed separately) to verify that LDAP is working properly. Start slapd with service slapd start, and then verify with the following ldapsearch command:

# ldapsearch -LLL -x -b'dc=example,dc=com'
No such object (32)

This result indicates that LDAP is running but the directory is empty. To import the puppet schema into this version of OpenLDAP, copy the puppet.schema from /usr/share/puppet/ext/ldap/puppet.schema to /etc/openldap/schema. Then create a configuration file named /tmp/puppet-ldap.conf with an include line pointing to that schema, as shown in the following snippet:

include /etc/openldap/schema/puppet.schema

Then run slaptest against that configuration file, specifying a temporary directory as storage for the configuration files created by slaptest, as shown in the following commands:

# mkdir /tmp/puppet-ldap
# slaptest -f puppet-ldap.conf -F /tmp/puppet-ldap/
config file testing succeeded

This will create an OLC structure in /tmp/puppet-ldap; the file we need is in /tmp/puppet-ldap/cn=config/cn=schema/cn={0}puppet.ldif. To import this file into our LDAP instance, we need to remove the ordering information (the braces and numbers ({0},{1},...) in this file). We also need to set the location for our schema, cn=schema,cn=config. All the lines after structuralObjectClass should be removed. The final version of the file will be in /tmp/puppet-ldap/cn=config/cn=schema/cn={0}puppet.ldif and will be as follows:

dn: cn=puppet,cn=schema,cn=config
objectClass: olcSchemaConfig
cn: puppet
olcAttributeTypes: ( 1.3.6.1.4.1.34380.1.1.3.10 NAME 'puppetClass' DESC 'Pu
ppet Node Class' EQUALITY caseIgnoreIA5Match SYNTAX 1.3.6.1.4.1.1466.115.121.
1.26 )
olcAttributeTypes: ( 1.3.6.1.4.1.34380.1.1.3.9 NAME 'parentNode' DESC 'Pupp
et Parent Node' EQUALITY caseIgnoreIA5Match SYNTAX 1.3.6.1.4.1.1466.115.121.1
.26 SINGLE-VALUE )
olcAttributeTypes: ( 1.3.6.1.4.1.34380.1.1.3.11 NAME 'environment' DESC 'Pu
ppet Node Environment' EQUALITY caseIgnoreIA5Match SYNTAX 1.3.6.1.4.1.1466.11
5.121.1.26 )
olcAttributeTypes: ( 1.3.6.1.4.1.34380.1.1.3.12 NAME 'puppetVar' DESC 'A va
riable setting for puppet' EQUALITY caseIgnoreIA5Match SYNTAX 1.3.6.1.4.1.146
6.115.121.1.26 )
olcObjectClasses: ( 1.3.6.1.4.1.34380.1.1.1.2 NAME 'puppetClient' DESC 'Pup
pet Client objectclass' SUP top AUXILIARY MAY ( puppetclass $ parentnode $ en
vironment $ puppetvar ) )

Now add this new schema to our instance using ldapadd as follows using the RootDN value cn=config:

# ldapadd -x -f cn\=\{0\}puppet.ldif -D'cn=config' -W
Enter LDAP Password: 
adding new entry "cn=puppet,cn=schema,cn=config"

Now we can start adding nodes to our LDAP installation. We'll need to add some containers and a top-level organization to the database before we can do that. Create a file named start.ldif with the following contents:

dn: dc=example,dc=com
objectclass: dcObject
objectclass: organization
o: Example
dc: example
dn: ou=hosts,dc=example,dc=com
objectclass: organizationalUnit
ou: hosts
dn: ou=production,ou=hosts,dc=example,dc=com
objectclass: organizationalUnit
ou: production

Tip

If you are unfamiliar with how LDAP is organized, review the information at http://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol#Directory_structure.

Now add the contents of start.ldif to the directory using ldapadd as follows:

# ldapadd -x -f start.ldif -D'cn=manager,dc=example,dc=com' -W
Enter LDAP Password: 
adding new entry "dc=example,dc=com"
adding new entry "ou=hosts,dc=example,dc=com"
adding new entry "ou=production,ou=hosts,dc=example,dc=com"

At this point, we have a container for our nodes at ou=production,ou=hosts,dc=example,dc=com; we can add an entry to our LDAP with the following LDIF, which we will name web_main_lp01.ldif:

dn: cn=web_main_lp01,ou=production,ou=hosts,dc=example,dc=com
objectclass: puppetClient
objectclass: device
environment: production
puppetClass: web
puppetClass: base
puppetvar: role='Production Web Server'

We then add this LDIF to the directory using ldapadd again, as shown in the following commands:

# ldapadd -x -f web_main_lp01.ldif -D'cn=manager,dc=example,dc=com' -W
Enter LDAP Password: 
adding new entry "cn=web_main_lp01,ou=production,ou=hosts,dc=example,dc=com"

With our entry in LDAP, we are ready to configure our worker nodes to look in LDAP for node definitions. Change /etc/puppet/puppet.conf to have the following lines in the [master] section:

node_terminus = ldap
ldapserver = ldap.example.com
ldapbase = ou=hosts,dc=example,dc=com

We are almost ready; we need ruby-ldap installed on the worker machine before Puppet can use LDAP to look up the node information. We can install it using the following steps:

#yum install ruby-ldap
Installed:
ruby-ldap-0.9.7-10.el6.x86_64

Now restart httpd to have the changes picked up. To convince yourself that the node definition is now coming from LDAP, modify the base class in /etc/puppet/modules/base/manifests/init.pp to include the role variable, as shown in the following snippet:

class base {
 file {'/etc/motd':
 mode => '0644',
 owner => '0',
 group => '0',
 content => inline_template("Role: <%= role %>\nManaged Node: <%= hostname %>\nManaged by Puppet version <%= puppetversion %>\n"),
 }
}

Tip

You will also need to open port 389, the standard LDAP port, on ldap.example.com, to allow the Puppet masters to query the LDAP machine.

Then run Puppet on web_main_lp01 and verify the contents of /etc/motd using the following commands:

# cat /etc/motd
Role: 'Production Web Server'
Managed Node: web_main_lp01
Managed by Puppet version 3.4.0

Keeping your class and variable information in LDAP makes sense if you already have all your nodes in LDAP for other purposes, such as DNS or DHCP. One potential drawback of this is that all of the class information for the node has to be stored within a single LDAP entry. It is useful to be able to apply classes to machines based on criteria. In the next section, we look at hiera, a system which allows for this type of criteria-based application.

Before starting the next section, comment out the LDAP ENC lines in /etc/puppet.conf as follows:

#  node_terminus = ldap
#  ldapserver = puppet.example.com
#  ldapbase = ou=hosts,dc=example,dc=com