In pig, you can pass a configuration from your pig script to pig UDF via UDFContext. For example,
// in pig script
SET my.conf dummy-conf
// in UDF java code
Configuration conf = UDFContext.getUDFContext().getJobConf();
String myConf = conf.get("my.conf");
So, is there a similar way to pass configuration from a hive script to a hive UDF? For example, if I have set MY_CONF='foobar' in a hive script, how can I retrieve that in a java UDF, which needs to consume the value of MY_CONF?
Instead of extending UDF class, you can try subclassing GenericUDF. This class has the following method you can override:
/**
* Additionally setup GenericUDF with MapredContext before initializing.
* This is only called in runtime of MapRedTask.
*
* @param context context
*/
public void configure(MapredContext context) {
}
MapredContext has a method just like UDFContext from Pig to retrieve the Job configuration. So you could just do the following:
@Override
public void configure(MapredContext context) {
Configuration conf = context.getJobConf();
}
As of hive 1.2 there are 2 approaches.
1. Overriding configure method from GenericUDF
@Override
public void configure(MapredContext context) {
super.configure(context);
someProp = context.getJobConf().get(HIVE_PROPERTY_NAME);
}
Above(1) won't work in all the cases. Works only in MapredContext. Every query has to be force map/reduce jobs, to do that set
set hive.fetch.task.conversion=minimal/none;
set hive.optimize.constant.propagation=false;
. with above properties set, you will hit major performance problems, especially for smaller queries.
2. Using SessionState
SessionState ss = SessionState.get();
if (ss != null) {
this.hiveConf = ss.getConf();
someProp = this.hiveConf.get(HIVE_PROPERTY_NAME);
LOG.info("Got someProp: " + someProp);
}
Go to hive command line
hive> set MY_CONF='foobar';
your variable should be listed when hitting the command
hive> set;
Now, consider you have below
Jar: MyUDF.jar
UDF calss: MySampleUDF.java which accepts a String value.
Table: employee
hive> ADD JAR /MyUDF.jar
hive> CREATE TEMPORARY FUNCTION testUDF AS 'youpackage.MySampleUDF';
hive> SELECT testUDF(${MY_CONF}) from employee;
there are lots of example, shared , so you can find all required details over google :).
A Small Example which was describe in shared link:
hive> ADD JAR assembled.jar;
hive> create temporary function hello as 'com.test.example.UDFExample';
hive> select hello(firstname) from people limit 10;
Please check link for reference which I normally Used to: Link1 Link2
