Setting up the Storm cluster

In this chapter, we will learn how to set up a three nodes Storm cluster, of which one node will be the active master node (Nimbus) and the other two will be worker nodes (supervisors).

The following is the deployment diagram of our three node Storm cluster:

The following are the steps that need to be performed to set up a three node Storm cluster:

1. Install and run the ZooKeeper cluster. The steps for installing ZooKeeper are mentioned in the previous section.
2. Download the latest stable Storm release from https://storm.apache.org/downloads.html; at the time of writing, the latest version is Storm 1.0.2.

1. Once you have downloaded the latest version, copy and unzip it in all three machines. Now, we will set the $STORM_HOME environment variable on each machine to make the setup easier. The $STORM_HOME environment contains the path of the Storm home folder (for example, export STORM_HOME=/home/user/storm-1.0.2).
2. Go to the $STORM_HOME/conf directory in the master nodes and add the following lines to the storm.yaml file:

storm.zookeeper.servers: 
- "zoo1" 
- "zoo2" 
- "zoo3" 
storm.zookeeper.port: 2181 
nimbus.seeds: "nimbus1,nimbus2" 
storm.local.dir: "/tmp/storm-data"  

1. Go to the $STORM_HOME/conf directory at each worker node and add the following lines to the storm.yaml file:

storm.zookeeper.servers: 
- "zoo1" 
- "zoo2" 
- "zoo3" 
storm.zookeeper.port: 2181 
nimbus.seeds: "nimbus1,nimbus2" 
storm.local.dir: "/tmp/storm-data" 
supervisor.slots.ports: 
- 6700 
- 6701 
- 6702 
- 6703  

1. Go to the $STORM_HOME directory at the master nodes and execute the following command to start the master daemon:

$> bin/storm nimbus &  

1. Go to the $STORM_HOME directory at each worker node (or supervisor node) and execute the following command to start the worker daemons:

$> bin/storm supervisor &