Using Troposphere to create a Python script for our template

We will first install the troposphere library. Again, we are demonstrating all of the outputs from a CentOS 7.x-based Linux distribution, but the process applies equally to all of the supported platforms mentioned. The following is the command to install the troposphere library:

$ pip install troposphere  

Once you have run the preceding command, you may encounter the following error:

....
setuptools_scm.version.SetuptoolsOutdatedWarning: your setuptools is too old (<12)     
-----------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-pW4aV4/cfn-flip/        

In order to fix the error, you can run the following command:

$ pip install -U setuptools

Collecting setuptools
   Downloading https://files.pythonhosted.org/packages/ff/f4/385715ccc461885f3cedf57a41ae3c12b5fec3f35cce4c8706b1a112a133/setuptools-40.0.0-py2.py3-none-any.whl (567kB)
        100% |████████████████████████████████| 573kB 22.2MB/s
Installing collected packages: setuptools
   Found existing installation: setuptools 0.9.8
      Uninstalling setuptools-0.9.8:
        Successfully uninstalled setuptools-0.9.8
Successfully installed setuptools-40.0.0 

Once the installation is complete, you can then create a new file called helloworld-cf-template.py.

We will start our file by importing a number of definitions from the troposphere module as follows:

"""Generating CloudFormation template."""

from troposphere import (
    Base64,
    ec2,
    GetAtt,
    Join,
    Output,
    Parameter,
    Ref,
    Template,
)  

We are also going to define a first variable that will make editing the code easier for the remainder of the book. This is because we will create new scripts by building on this initial template:

ApplicationPort = "3000"  

From a code standpoint, the first thing we will do is initialize a Template variable. By the end of our script, the template will contain the entire description of our infrastructure and we will be able to simply print its output to get our CloudFormation template:

t = Template() 

Throughout this book, we will create and run several CloudFormation templates concurrently. To help us identify what's in a given stack, we have the ability to provide a description. After the creation of the template, add the description as follows:

add_description("Effective DevOps in AWS: HelloWorld web application") 

When we launched EC2 instances using the web command-line interface, we selected which key-pair to use in order to gain SSH access to the host. In order to not lose this ability, the first thing our template will have is a parameter to offer the CloudFormation user the ability to select which key-pair to use when launching the EC2 instance. To do that, we are going to create a Parameter object and initialize it by providing an identifier, a description, a parameter type, a description of the parameter type, and a constraint description to help make the right decision when we launch the stack. In order for this parameter to exist in our final template, we will also use the add_parameter() function defined in the template class:

t.add_parameter(Parameter(
    "KeyPair",
    Description="Name of an existing EC2 KeyPair to SSH",
    Type="AWS::EC2::KeyPair::KeyName",
    ConstraintDescription="must be the name of an existing EC2 KeyPair.",
))

The next thing we will look at is the security group. We will proceed exactly as we did for our KeyPair parameter. We want to open up SSH/22 and TCP/3000 to the world. Port 3000 was defined in the ApplicationPort variable declared earlier. In addition, this time, the information defined isn't a parameter like before, but a resource. Consequently, we will add that new resource using the add_resource() function as follows:

t.add_resource(ec2.SecurityGroup(
    "SecurityGroup",
    GroupDescription="Allow SSH and TCP/{} access".format(ApplicationPort),
    SecurityGroupIngress=[
        ec2.SecurityGroupRule(
            IpProtocol="tcp",
            FromPort="22",
            ToPort="22",
            CidrIp="0.0.0.0/0",
        ),
        ec2.SecurityGroupRule(
            IpProtocol="tcp",
            FromPort=ApplicationPort,
            ToPort=ApplicationPort,
            CidrIp="0.0.0.0/0",
        ),
    ],
))

In our next section, we will replace the need to log on to our EC2 instance and install the helloworld.js file and its init scripts by hand. To do so, we will take advantage of the UserData features that EC2 offers. When you create an EC2 instance, the UserData optional parameter gives you the ability to provide a set of commands to run once the virtual machine has spawned up (you can read more on this topic at http://amzn.to/1VU5b3s). One of the constraints of the UserData parameter is that the script must be base64-encoded in order to be added to our API call.

We are going to create a small script to reproduce the steps that we went through in Chapter 2, Deploying Your First Web Application. Here, we will encode, deploy our first web application deployment step in base-64 and store it in a variable called ud. Note that installing the application in the home directory of ec2-user isn't very clean. For now, we are trying to stay consistent with what we did in Chapter 2, Deploying Your First Web Application. We will fix that in Chapter 5, Adding Continuous Integration and Continuous Deployment, as we improve our deployment system:

ud = Base64(Join('\n', [
    "#!/bin/bash",
    "sudo yum install --enablerepo=epel -y nodejs",
    "wget http://bit.ly/2vESNuc -O /home/ec2-user/helloworld.js",
    "wget http://bit.ly/2vVvT18 -O /etc/init/helloworld.conf",
    "start helloworld"
]))

We will now focus on the main resource of our template, which is our EC2 instance. The creation of the instance requires providing a name for identifying the resource, an image ID, an instance type, a security group, the key-pair to use for the SSH access, and the user data. In order to keep things simple, we will hardcode the AMI ID (ami-cfe4b2b0) and instance type (t2.micro).

The remaining pieces of information needed to create our EC2 instances are the security group information and the KeyPair name, which we collected previously by defining a parameter and a resource. In CloudFormation, you can refer to pre-existing subsections of your template by using the Ref keyword. In Troposphere, this is done by calling the Ref() function. As before, we will add the resulting output to our template with the help of the add_resource function:

...
t.add_resource(ec2.Instance(
    "instance",
    ImageId="ami-cfe4b2b0",
    InstanceType="t2.micro",
    SecurityGroups=[Ref("SecurityGroup")],
    KeyName=Ref("KeyPair"),
    UserData=ud,
)) 
...

In the last section of our script, we will focus on producing the Outputs section of the template that gets populated when CloudFormation creates a stack. This selection allows you to print out useful information that was computed during the launch of the stack. In our case, there are two useful pieces of information—the URL to access our web application, and the public IP address of the instance, so that we can SSH into it if we want to. In order to retrieve such information, CloudFormation uses the Fn::GetAtt  function. In Troposphere, this is translated into the  GetAtt() function:

...
t.add_output(Output(
    "InstancePublicIp",
    Description="Public IP of our instance.",
    Value=GetAtt("instance", "PublicIp"),
))

t.add_output(Output(
    "WebUrl",
    Description="Application endpoint",
    Value=Join("", [
        "http://", GetAtt("instance", "PublicDnsName"),
        ":", ApplicationPort
    ]),
)) 
...

At that point, we can make our script output the final result of the template we generated:

print t.to_json() 

The script is now complete. We can save this and quit our editor. The file created should look like the file at the following link:  https://raw.githubusercontent.com/yogeshraheja/Effective-DevOps-with-AWS/master/Chapter03/EffectiveDevOpsTemplates/helloworld-cf-template-part-1.py.

We can now run our script, giving it the proper permissions and generating the CloudFormation template by saving the output of our script in a file as follows:

$ python helloworld-cf-template.py > helloworld-cf.template