ApacheSparkisanopensourceparallel-processingframeworkthathasbeenaroundforquitesometimenow.OneofthemanyusesofApacheSparkisfordataanalyticsapplicationsacrossclusteredcomputers.Inthisbook,youwillnotonlylearnhowtouseSparkandthePythonAPItocreatehigh-performanceanalyticswithbigdata,butalsodiscovertechniquesfortesting,immunizing,andparallelizingSparkjobs.Youwilllearnhowtosourcedatafromallpopulardatahostingplatforms,includingHDFS,Hive,JSON,andS3,anddealwithlargedatasetswithPySparktogainpracticalbigdataexperience.Thisbookwillhelpyouworkonprototypesonlocalmachinesandsubsequentlygoontohandlemessydatainproductionandatscale.ThisbookcoversinstallingandsettingupPySpark,RDDoperations,bigdatacleaningandwrangling,andaggregatingandsummarizingdataintousefulreports.YouwillalsolearnhowtoimplementsomepracticalandproventechniquestoimprovecertainaspectsofprogrammingandadministrationinApacheSpark.Bytheendofthebook,youwillbeabletobuildbigdataanalyticalsolutionsusingthevariousPySparkofferingsandalsooptimizethemeffectively.