Installing the prerequisites

In Linux, you will need to install all prerequisites through the package manager so they can be updated, removed, and managed in a much cleaner way. Overall, you will find two major flavors for Linux that each have different package management tools; they are as follows:

• RedHat Enterprise, Fedora, and CentOS primarily deal with rpm and they use yum and rpm
• Debian and Ubuntu use .deb for package management, and you can use apt-get or dpkg

In addition to the tools available on the command-line interface, you can also use user interface-based package management tools such as the software center or package manager, which are provided through the admin functionality of the mentioned operating systems. Before you start working on prerequisites, you must first update your local package manager database with the latest updates from source with the following command:

hadoop@base0:/$ sudo apt-get update

The update will take some time depending on the state of your OS. Once the update is complete, you may need to install an SSH client on your system. Secure Shell is used to connect Hadoop nodes with each other; this can be done with the following command:

hadoop@base0:/$ sudo apt-get install ssh

Once SSH is installed, you need to test whether you have the SSH server and client set up correctly. You can test this by simply logging in to the localhost using the SSH utility, as follows:

hadoop@base0:/$ ssh localhost

You will then be asked for the user's password that you typed earlier, and if you log in successfully, the setup has been successful. If you get a 'connection refused' error relating to port 22, you may need to install the SSH server on your system, which can be done with the following command:

hadoop@base0:/$ sudo apt-get install openssh-server

Next, you will need to install JDK on your system. Hadoop requires JDK version 1.8 and above. (Please visit this link for older compatible Java versions.) Most of the Linux installations have JDK installed by default, however, you may need to look for compatibility. You can check the current installation on your machine with the following command:

hadoop@base0:/$ sudo apt list | grep openjdk

To remove an older installation, use the following command:

hadoop@base0:/$ sudo apt-get remove <old-jdk>

To install JDK 8, use the following command:

hadoop@base0:/$ sudo apt-get install openjdk-8-jdk

You need to ensure that your JAVA_HOME environment variable is set correctly in the Hadoop environment file, which is found in $HADOOP_HOME/etc/hadoop/hadoop-env.sh. Make sure that you add the following entry:

export JAVA_HOME= <location-of-java-home>