- Hands-On Artificial Intelligence for IoT
- Amita Kapoor
- 285字
- 2021-07-02 14:02:02
HDFS
HDFS is a popular storage and access method for storing and retrieving data files for IoT solutions. The HDFS format can hold large amounts of data in a reliable and scalable manner. Its design is based on the Google File System (https://ai.google/research/pubs/pub51). HDFS splits individual files into fixed-size blocks that are stored on machines across the cluster. To ensure reliability, it replicates the file blocks and distributes them across the cluster; by default, the replication factor is 3. HDFS has two main architecture components:
- The first, NodeName, stores the metadata for the entire filesystem, such as filenames, their permissions, and the location of each block of each file.
- The second, DataNode (one or more), is where file blocks are stored. It performs Remote Procedure Calls (RPCs) using protobufs.
RPC is a protocol that one program can use to request a service from a program located on another computer on a network without having to know the network's details. A procedure call is also sometimes known as a function call or a subroutine call.
There are many options for programmatically accessing HDFS in Python, such as snakebite, pyarrow, hdfs3, pywebhdfs, hdfscli, and so on. In this section, we will focus mainly on libraries that provide native RPC client interfaces and work with Python 3.
- 精通MATLAB神經網絡
- 基于LabWindows/CVI的虛擬儀器設計與應用
- Mastering Salesforce CRM Administration
- 機器人智能運動規劃技術
- 最簡數據挖掘
- Google SketchUp for Game Design：Beginner's Guide
- INSTANT Adobe Story Starter
- Mastering OpenStack（Second Edition）
- C#求職寶典
- 機床電氣控制與PLC
- Machine Learning with Spark（Second Edition）
- 人工智能云平臺：原理、設計與應用
- 微機組裝與維護教程
- 軟件測試設計
- 當產品經理遇到人工智能