官术网_书友最值得收藏!

Finding the sum of all even numbers in an RDD of integers

Let's first create an RDD of integers as follows:

scala>valintRDD = sc.parallelize(Array(1,4,5,6,7,10,15))
intRDD: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[0] at parallelize at <console>:24

The next step is to filter all the even elements in this RDD. So, we will execute a filter operation on the RDD, as follows:

scala>valevenNumbersRDD=intRDD.filter(i => (i%2==0))
evenNumbersRDD: org.apache.spark.rdd.RDD[Int] = MapPartitionsRDD[3] at filter at <console>:26

The preceding operation will fetch those elements for which 2 is a factor, that is, even elements .

Now, we will sum all the elements of this RDD as follows:

scala>val sum =evenNumbersRDD.sum
sum: Double = 20.0
scala> sum
res5: Double = 20.0

It is shown in the following screenshot:

主站蜘蛛池模板: 淮阳县| 宜阳县| 高安市| 双峰县| 灵石县| 甘洛县| 佛学| 临朐县| 尼勒克县| 信阳市| 赤水市| 保德县| 隆回县| 凉山| 商都县| 湛江市| 临泽县| 阜新市| 平湖市| 桐城市| 龙州县| 西吉县| 昌平区| 盖州市| 达日县| 海晏县| 游戏| 庆城县| 旺苍县| 凤凰县| 抚远县| 阿巴嘎旗| 南开区| 永宁县| 常德市| 探索| 临泉县| 西乡县| 兴宁市| 巴林左旗| 石渠县|