- Hands-On Artificial Intelligence for IoT
- Amita Kapoor
- 154字
- 2021-07-02 14:02:02
Using hdfs3 with HDFS
hdfs3 is a lightweight Python wrapper around the C/C++ libhdfs3 library. It allows us to use HDFS natively from Python. To start, we first need to connect with the HDFS NameNode; this is done using the HDFileSystem class:
from hdfs3 import HDFileSystem
hdfs = HDFileSystem(host = 'localhost', port=8020)
This automatically establishes a connection with the NameNode. Now, we can access a directory listing using the following:
print(hdfs.ls('/tmp'))
This will list all the files and directories in the tmp folder. You can use functions such as mkdir to make a directory and cp to copy a file from one location to another. To write into a file, we open it first using the open method and use write:
with hdfs.open('/tmp/file1.txt','wb') as f:
f.write(b'You are Awesome!')
Data can be read from the file:
with hdfs.open('/tmp/file1.txt') as f:
print(f.read())
You can learn more about hdfs3 from its documentation: https://media.readthedocs.org/pdf/hdfs3/latest/hdfs3.pdf.
推薦閱讀
- 腦動力:Linux指令速查效率手冊
- 軟件架構設計
- 3D Printing with RepRap Cookbook
- 輕松學Java
- IoT Penetration Testing Cookbook
- 工業機器人工程應用虛擬仿真教程:MotoSim EG-VRC
- 樂高創意機器人教程(中級 下冊 10~16歲) (青少年iCAN+創新創意實踐指導叢書)
- Android游戲開發案例與關鍵技術
- Azure PowerShell Quick Start Guide
- 深度學習與目標檢測
- Learning ServiceNow
- 工業機器人實操進階手冊
- 基于RPA技術財務機器人的應用與研究
- 30天學通Java Web項目案例開發
- Oracle 11g基礎與提高