Creating and running a Hadoop Mapreduce Jar file by using HBase

Question

I have a MapReduce implementation where the input for the Mapper is an HBase scan result (one map for each row) and the Reduce phase stores the output directly in a specific HBase table.

In order to specify mapper and reducer parameters I'm using the TableMapReduceUtil helper class as follows:

TableMapReduceUtil.initTableMapperJob(
              "Simulations_Data",       // input HBase table name
              scan_job_DMV,                 // Scan instance to control CF and attribute selection <key, value>
              HBaseMapper.class,        // mapper
              TimestepNode.class,           // mapper output key
              DoubleArrayWritable.class,    // mapper output value
              to_return);


TableMapReduceUtil.initTableReducerJob(
              "SimulationsResults",     // output table
              HBaseReducer.class,       // reducer class
              to_return);

The working Environment is hbase-0.98.4-hadoop2 and hadoop-2.6.0

If I run the code by using Eclipse (Version: Luna Service Release 2 - 4.4.2), everything works perfectly. But, if I generate a Runnable JAR file from Eclipse (with the option Package required lib, into generated JAR) and try to run it from the terminal, the following error occurs:

Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: rsrc:hbase-common-0.98.11-hadoop2.jar
    at org.apache.hadoop.fs.Path.initialize(Path.java:206)
    at org.apache.hadoop.fs.Path.<init>(Path.java:172)
    at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar(TableMapReduceUtil.java:823)
    at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:776)
    at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addHBaseDependencyJars(TableMapReduceUtil.java:692)
    at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:737)
    at D2C_mapreduce_main.get_DMV_Job(D2C_mapreduce_main.java:130)
    at D2C_mapreduce_main.main(D2C_mapreduce_main.java:102)
    ... 5 more
Caused by: java.net.URISyntaxException: Relative path in absolute URI: rsrc:hbase-common-0.98.11-hadoop2.jar
    at java.net.URI.checkPath(URI.java:1804)
    at java.net.URI.<init>(URI.java:752)
    at org.apache.hadoop.fs.Path.initialize(Path.java:203)
    ... 12 more

This problem has been partially addressed here:

But still, maybe for the lack of resources online, I think it can represent an open problem for the community.

Creating and running a Hadoop Mapreduce Jar file by using HBase

0 Answers0