官术网_书友最值得收藏!

Time for action – using the Writable wrapper classes

Let's write a class to show some of these wrapper classes in action:

  1. Create the following as WritablesTest.java:
    import org.apache.hadoop.io.* ;
    import java.util.* ;
    
    public class WritablesTest
    {
        public static class IntArrayWritable extends ArrayWritable
        {
            public IntArrayWritable()
            {
                super(IntWritable.class) ;
            }
        }
        
        public static void main(String[] args)
        {
    System.out.println("*** Primitive Writables ***") ;
            BooleanWritable bool1 = new BooleanWritable(true) ;
            ByteWritable byte1 = new ByteWritable( (byte)3) ;
            System.out.printf("Boolean:%s Byte:%d\n", bool1, byte1.get()) ;
            
            IntWritable i1 = new IntWritable(5) ;
            IntWritable i2 = new IntWritable( 17) ;
            System.out.printf("I1:%d I2:%d\n", i1.get(), i2.get()) ;
            i1.set(i2.get()) ;
            System.out.printf("I1:%d I2:%d\n", i1.get(), i2.get()) ;
            Integer i3 = new Integer( 23) ;
            i1.set( i3) ;
            System.out.printf("I1:%d I2:%d\n", i1.get(), i2.get()) ;
     
    System.out.println("*** Array Writables ***") ;       
            ArrayWritable a = new ArrayWritable( IntWritable.class) ;
            a.set( new IntWritable[]{ new IntWritable(1), new IntWritable(3), new IntWritable(5)}) ;
            
            IntWritable[] values = (IntWritable[])a.get() ;
            
            for (IntWritable i: values)
            System.out.println(i) ;
            
            IntArrayWritable ia = new IntArrayWritable() ;
            ia.set( new IntWritable[]{ new IntWritable(1), new IntWritable(3), new IntWritable(5)}) ;
            
            IntWritable[] ivalues = (IntWritable[])ia.get() ;
            
            ia.set(new LongWritable[]{new LongWritable(1000l)}) ;
     
    System.out.println("*** Map Writables ***") ;       
            MapWritable m = new MapWritable() ;
            IntWritable key1 = new IntWritable(5) ;
            NullWritable value1 = NullWritable.get() ;
            m.put(key1, value1) ;
            System.out.println(m.containsKey(key1)) ;
            System.out.println(m.get(key1)) ;
            m.put(new LongWritable(1000000000), key1) ;
            Set<Writable> keys = m.keySet() ;
            
            for(Writable w: keys)
            System.out.println(w.getClass()) ;
        }
    }
  2. Compile and run the class, and you should get the following output:
    *** Primitive Writables ***
    Boolean:true Byte:3
    I1:5 I2:17
    I1:17 I2:17
    I1:23 I2:17
    *** Array Writables ***
    1
    3
    5
    *** Map Writables ***
    true
    (null)
    class org.apache.hadoop.io.LongWritable
    class org.apache.hadoop.io.IntWritable
    

What just happened?

This output should be largely self-explanatory. We create various Writable wrapper objects and show their general usage. There are several key points:

  • As mentioned, there is no type-safety beyond Writable itself. So it is possible to have an array or map that holds multiple types, as shown previously.
  • We can use autounboxing, for example, by supplying an Integer object to methods on IntWritable that expect an int variable.
  • The inner class demonstrates what is needed if an ArrayWritable class is to be used as an input to a reduce function; a subclass with such a default constructor must be defined.

Other wrapper classes

  • CompressedWritable: This is a base class to allow for large objects that should remain compressed until their attributes are explicitly accessed
  • ObjectWritable: This is a general-purpose generic object wrapper
  • NullWritable: This is a singleton object representation of a null value
  • VersionedWritable: This is a base implementation to allow writable classes to track versions over time

Have a go hero – playing with Writables

Write a class that exercises the NullWritable and ObjectWritable classes in the same way as it does in the previous examples.

Making your own

As you have seen from the Writable and Comparable interfaces, the required methods are pretty straightforward; don't be afraid of adding this functionality if you want to use your own custom classes as keys or values within a MapReduce job.

主站蜘蛛池模板: 浦县| 肃南| 仁怀市| 丽水市| 名山县| 边坝县| 建阳市| 霍邱县| 晋城| 威信县| 贵州省| 太湖县| 化德县| 庄浪县| 舞阳县| 铜陵市| 阿坝| 延川县| 顺昌县| 武功县| 临西县| 华安县| 宽甸| 龙山县| 原阳县| 东乌| 哈尔滨市| 丰原市| 宁阳县| 靖远县| 汤原县| 淳化县| 平遥县| 吉林市| 昆山市| 七台河市| 仲巴县| 瓮安县| 疏附县| 汤原县| 娄烦县|