Configure parallelism at the code level

Storm provides an API to set the number of worker processes, number of executors, and number of tasks at the code level. The following section shows how we can configure parallelism at the code level.

We can set the number of worker processes at the code level by using the setNumWorkers method of the org.apache.storm.Config class. Here is the code snippet to show these settings in practice:

Config conf = new Config(); 
conf.setNumWorkers(3); 

In the previous chapter, we configured the number of workers as three. Storm will assign the three workers for the SampleStormTopology and SampleStormClusterTopology topology.

We can set the number of executors at the code level by passing the parallelism_hint argument in the setSpout(args,args,parallelism_hint) or setBolt(args,args,parallelism_hint) methods of the org.apache.storm.topology.TopologyBuilder class. Here is the code snippet to show these settings in practice:

builder.setSpout("SampleSpout", new SampleSpout(), 2); 
// set the bolt class 
builder.setBolt("SampleBolt", new SampleBolt(), 4).shuffleGrouping("SampleSpout"); 

In the previous chapter, we set parallelism_hint=2 for SampleSpout and parallelism_hint=4 for SampleBolt. At the time of execution, Storm will assign two executors for SampleSpout and four executors for SampleBolt.

We can configure the number of tasks that can execute inside the executors. Here is the code snippet to show these settings in practice:

builder.setSpout("SampleSpout", new SampleSpout(), 2).setNumTasks(4); 

In the preceding code, we have configured the two executors and four tasks of SampleSpout. For SampleSpout, Storm will assign two tasks per executor. By default, Storm will run one task per executor if the user does not set the number of tasks at the code level.