- Hadoop 2.x Administration Cookbook
- Gurmukh Singh
- 379字
- 2021-07-09 20:10:27
Setting up Namenode metadata location
The most critical component of Hadoop is Namenode, and it is important to safeguard the information it stores. It stores metadata, which is a combination of namespace and inode structure.
All the steps are to be performed as the hadoop user. It is expected that the user has gone through Chapter 1, Hadoop Architecture and Deployment and understands the uses and function of Namenode.
Getting ready
You are going to need a preinstalled Hadoop as discussed in Chapter 1, Hadoop Architecture and Deployment. In the following recipes, we will configure the parameters for a copy of Hadoop that is already installed.
How to do it...
sshto the Namenode, which in this case isnn1.cluster1.com.- Navigate to the
/opt/cluster/hadoop/etc/hadoopdirectory. This is the directory where we installed Hadoop in the first chapter. If the user has installed it at a different location, then navigate to this directory. - Configure the
dfs.namenode.name.dirparameter, which defines the location where Namenode will persist its metadata. - See the following screenshot for the contents of the file:
- It is good practice to have the
/data/namenodemount point on a different disk drive, rather than from a disk on which the OS is installed. - In production, there will always be more than one directory to store the metadata, as shown in the following screenshot:
- Once the changes are made, save the file and restart the
namenodedaemon. - We need to format the Namenode so that all the directories listed are in sync and initialized.
- If the user already had a running cluster, then copy the contents across the directories and do not format the Namenode, as it will result in the loss of data.
How it works...
With steps 3 to 5, the user will be able to configure the Namenode metadata store location. Usually, in production, one of the common separated lists of directories is a cross mount from the NFS server. This is done to make sure that, if the Namenode server goes down, we can still access the data by quickly mounting it on another machine.
Please start the Datanode daemons on all nodes configured to run as Datanodes. If the Datanodes were initially connected to another Namenode, then the Datanode directories specified using dfs.datanode.data.dir must be cleaned.
- PowerShell 3.0 Advanced Administration Handbook
- 傳感器技術實驗教程
- 圖解PLC控制系統梯形圖和語句表
- Mastering Salesforce CRM Administration
- 大學C/C++語言程序設計基礎
- 數據庫系統原理及應用教程（第5版)
- Python：Data Analytics and Visualization
- INSTANT Munin Plugin Starter
- Mastering GitLab 12
- 機床電氣控制與PLC
- Natural Language Processing and Computational Linguistics
- 網絡安全概論
- Windows 7來了
- 暗戰強人：黑客及反黑客工具快速精通
- 工業機器人設計與實例詳解