官术网_书友最值得收藏!

Spark graph processing

Graph processing is another very important topic when it comes to data analysis. In fact, a majority of problems can be expressed as a graph.

A graph is basically a network of items and their relationships to each other. Items are called nodes and relationships are called edges. Relationships can be directed or undirected. Relationships, as well as items, can have properties. So a map, for example, can be represented as a graph as well. Each city is a node and the streets between the cities are edges. The distance between the cities can be assigned as properties on the edge.

The Apache Spark GraphX module allows Apache Spark to offer fast big data in-memory graph processing. This allows you to run graph algorithms at scale.

One of the most famous algorithms, for example, is the traveling salesman problem. Consider the graph representation of the map mentioned earlier. A salesman has to visit all cities of a region but wants to minimize the distance that he has to travel. As the distances between all the nodes are stored on the edges, a graph algorithm can actually tell you the optimal route. GraphX is able to create, manipulate, and analyze graphs using a variety of built-in algorithms.

It introduces two new data types to support graph processing in Spark--VertexRDD and EdgeRDD--to represent graph nodes and edges. It also introduces graph processing algorithms, such as PageRank and triangle processing. Many of these functions will be examined in Chapter 11, Apache Spark GraphX and Chapter 12, Apache Spark GraphFrames.

主站蜘蛛池模板: 资溪县| 灌云县| 邵阳市| 崇文区| 绥滨县| 平武县| 合川市| 思茅市| 石渠县| 海门市| 中阳县| 靖州| 长汀县| 科技| 达尔| 石台县| 隆林| 东明县| 乌兰察布市| 开原市| 临高县| 广饶县| 汉阴县| 平阴县| 永德县| 罗甸县| 密山市| 静乐县| 温州市| 荥经县| 镶黄旗| 汝州市| 渑池县| 芷江| 托里县| 运城市| 鹤岗市| 托克逊县| 灵石县| 宣化县| 富蕴县|