Working on the cluster

This section will cover some of the tools and techniques to monitor and debug the application in the distributed environment. We will also look at some of the options to apply changes to the application without rebuilding or packaging it. The tools we use in this section are standard components of Apex, Hadoop, and the operating system (nothing distribution or vendor specific).

Let's begin with some of the basic tools and commands that will allow us to gather information. YARN provides a basic web interface to look at information about running applications and container processes. Examples are based on the local Docker environment which was discussed earlier. The tools are all standard and available when working with a different cluster setup as well, although machine addresses and access may differ.