UI in Spark

Spark provides a web interface which can be used to monitor jobs, see the environment, and run SQL commands.

SparkContext launches a web UI on port 4040 that displays useful information about the application. This includes the following:

• A list of scheduler stages and tasks
• A summary of RDD sizes and memory usage
• Environmental information
• Information about the running executors

This interface can be accessed by going to http://<driver-node>:4040 in a web browser. If multiple SparkContexts are running on the same host, they will bind to ports beginning with port 4040 (4041, 4042, and so on).

The following screenshots display some of the information provided by the Web UI: