官术网_书友最值得收藏!

Finding the sum of all even numbers in an RDD of integers

Let's first create an RDD of integers as follows:

scala>valintRDD = sc.parallelize(Array(1,4,5,6,7,10,15))
intRDD: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[0] at parallelize at <console>:24

The next step is to filter all the even elements in this RDD. So, we will execute a filter operation on the RDD, as follows:

scala>valevenNumbersRDD=intRDD.filter(i => (i%2==0))
evenNumbersRDD: org.apache.spark.rdd.RDD[Int] = MapPartitionsRDD[3] at filter at <console>:26

The preceding operation will fetch those elements for which 2 is a factor, that is, even elements .

Now, we will sum all the elements of this RDD as follows:

scala>val sum =evenNumbersRDD.sum
sum: Double = 20.0
scala> sum
res5: Double = 20.0

It is shown in the following screenshot:

主站蜘蛛池模板: 兴安盟| 临江市| 南陵县| 西乡县| 聊城市| 将乐县| 建湖县| 南京市| 正镶白旗| 高邑县| 炎陵县| 兖州市| 平湖市| 阳高县| 台湾省| 新津县| 宁德市| 稷山县| 九龙坡区| 桐庐县| 会泽县| 临猗县| 迁安市| 大冶市| 满城县| 新安县| 财经| 桂阳县| 顺平县| 宁城县| 洛阳市| 眉山市| 墨竹工卡县| 楚雄市| 天台县| 保定市| 饶阳县| 宝兴县| 大新县| 儋州市| 武功县|