1

I want to be able to use HBase with spring. The documentation I can see has something that says:

<hdp:configuration>
        fs.default.name=${hd.fs}
        mapred.job.tracker=${mapred.job.tracker}
        hbase.zookeeper.quorum=${hbase.zookeeper.quorum}
</hdp:configuration>

How can I programatically use this in my Application.java? There appears to be no sample or documentation on how to use HBaseTemplate to connect to hbase. Looking at the following repository as recommended in another post has no hbase-related examples:

https://github.com/spring-projects/spring-hadoop-samples

Rolando
  • 58,640
  • 98
  • 266
  • 407
  • Related question: [How to use hbase with Spring Boot using Java instead of XML?](http://stackoverflow.com/questions/24795097/how-to-use-hbase-with-spring-boot-using-java-instead-of-xml) – blong Oct 22 '15 at 02:20

2 Answers2

0

Currently I haven't found annotation based configuration for hdp:configuration, but instead for my Spring-Hadoop Project I have created a xml for haddop configuration as hadoop-context.xml and imported it in Config.java like below

    @Configuration
    @ImportResource ( "hadoop-context.xml")
    public class Config
    {

     @Autowired
     private org.apache.hadoop.conf.Configuration hadoopConfiguration;
    }

Most probably it will help in your case to solve the issue.

Swaraj
  • 589
  • 4
  • 15
0

I know the question is old but I just ran into this and was able to resolve. Implement SpringHadoopConfigurerAdapter like:

@Configuration
@EnableHadoop
public class HadoopConfiguration extends SpringHadoopConfigurerAdapter {
    @Autowired
    private ResourceLoader resourceLoader;

    @Override
    public void configure(HadoopConfigConfigurer config) throws Exception {

        org.apache.hadoop.conf.Configuration conf = getHadoopConfig("classpath:/mapred-site.xml");

        config
                .withResources()
                .resource("classpath:/yarn-site.xml")
                .resource("classpath:/hbase-site.xml")
                .resource("classpath:/mapred-site.xml")
                .and()
                .withProperties()
                .property("mapreduce.app-submission.cross-platform", "true")
                .property("mapreduce.framework.name", "yarn")
                .property("mapreduce.application.framework.path", "")
                .property("mapreduce.map.log.level", "INFO")
                .property("mapreduce.reduce.log.level", "INFO")
                .property("hbase.client.scanner.caching", "1")
                .property("hbase.client.scanner.timeout.period", "300000")
                .property("hbase.rpc.timeout", "300000");

    }

    private org.apache.hadoop.conf.Configuration getHadoopConfig(String mrSiteFile) throws IOException {
        Resource mapRedSite = resourceLoader.getResource(mrSiteFile);
        org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
        conf.addResource(mapRedSite.getInputStream());
        return conf;
    }
}

Then you'll be able to inject your HadoopConfiguration:

@Autowired
org.apache.hadoop.conf.Configuration hadoopConfiguration;
Litz
  • 36
  • 3