- Mastering Hadoop
- Sandeep Karanth
- 263字
- 2021-08-06 19:53:01
Chapter 3. Advanced Pig
Running Java MapReduce jobs on Hadoop provides the most flexibility with the least abstraction. However, abstractions are necessary to infer patterns, accomplish common data manipulation tasks, reduce complexity, and flatten the learning curve. Pig is a platform that provides a framework and high-level abstractions to build MapReduce programs for Hadoop. It has a scripting language called Pig Latin. Pig Latin can be compared to SQL in terms of operator capabilities.
Developed at Yahoo! around the year 2006, Pig was used as a framework to specify ad hoc MapReduce workflows. In the following year, it was moved to Apache Software Foundation. The latest release of Pig is 0.12.1.
Tip
The official release of Pig is currently incompatible with Hadoop 2.2.0. It expects libraries from Hadoop 1.2.1. Running any Pig script fails, with the following exception:
Unexpected System Error Occured: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected.
Fixing this requires a recompile of the Pig binaries. Run the following command and replace the newly generated pig.jar and pig-withouthadoop.jar files:
ant clean jar-all -Dhadoopversion=23
In this chapter, we will look at the advanced features of Pig by:
- Looking at how Pig is different when compared to SQL
- Analyzing how Pig Latin scripts are translated to MapReduce programs
- Delving into the advanced relational operators that Pig supports; we will delve deep into these relational operators and look at their applications with examples
- Studying ways to extend Pig beyond its off-the-shelf capabilities using User-defined Functions or UDFs that can implement a variety of interfaces; we will examine some of these interfaces
- Word 2003、Excel 2003、PowerPoint 2003上機指導與練習
- 西門子S7-200 SMART PLC從入門到精通
- Effective DevOps with AWS
- Visual Basic從初學到精通
- 工業機器人現場編程（FANUC）
- Maya極速引擎：材質篇
- 精通數據科學算法
- 菜鳥起飛系統安裝與重裝
- OpenStack Cloud Computing Cookbook
- Pentaho Analytics for MongoDB
- INSTANT Puppet 3 Starter
- Machine Learning Algorithms（Second Edition）
- Flink原理與實踐
- 3ds Max造型表現藝術
- 傳感器原理及實用技術