Spark Core

Spark Core is the main component of Spark. Spark Core defines the following:

• The basic components, such as RDD and DataFrames
• The APIs available to perform operations on these basic abstractions
• Shared or distributed variables, such as broadcast variables and accumulators

We shall look at them in more detail in the upcoming chapters.

Spark Core also defines all the basic functionalities, such as task management, memory management, basic I/O functionalities, and more. It’s a good idea to have a look at the Spark code on GitHub (https://github.com/apache/spark).