Stack Overflow
Stack Overflow

Questions tagged [spring-data-hadoop]

Spring for Apache Hadoop is an open-source project that provides unified configuration model and easy to use APIs for using HDFS, MapReduce, Pig, and Hive as well as developing and deploying YARN applications.

Spring for Apache Hadoop simplifies developing Apache Hadoop by providing a unified configuration model and easy to use APIs for using HDFS, MapReduce, Pig, and Hive. It also provides integration with other Spring ecosystem project such as Spring Integration and Spring Batch enabling you to develop solutions for big data ingest/export and Hadoop workflow orchestration.

Home page: http://projects.spring.io/spring-hadoop/

GitHub repo: https://github.com/spring-projects/spring-hadoop

47 questions
27
votes
5 answers

Triggering spark jobs with REST

I have been of late trying out apache spark. My question is more specific to trigger spark jobs. Here I had posted question on understanding spark jobs. After getting dirty on jobs I moved on to my requirement. I have a REST end point where I expose…
chaosguru
  • 1,933
  • 4
  • 30
  • 44
7
votes
0 answers

Why is spring-hadoop end-of-lifed and is there a successor?

According to the README in spring-hadoop, the project has reached an End-Of-Life: NOTICE: The Spring for Apache Hadoop project has reached End-Of-Life status on April 5th, 2019. The final Spring for Apache Hadoop 2.5.0 release was built using…
carlspring
  • 31,231
  • 29
  • 115
  • 197
7
votes
2 answers

How to submit multiple Spark applications in parallel without spawning separate JVMs?

The problem is that you need to launch separate JVM to create separate session with different number of RAM per job. How to submit few Spark applications simultaneously without manually spawning separate JVMs? My app is run on single server, within…
VB_
  • 45,112
  • 42
  • 145
  • 293
4
votes
0 answers

Spring Hadoop Java config

What is the current state of JavaConfig support? As I see it's possible to set up Hadoop configuration this way:…
Yaroslav Yermilov
  • 86
  • 3
4
votes
2 answers

Use Hbase with spring boot

Is there a sample project show how to use hbase in spring boot ? I read this link How to use hbase with Spring Boot using Java instead of XML?. But the instruction is not detail enough.
SieuCau
  • 195
  • 1
  • 2
  • 15
4
votes
2 answers

maven-shade-plugin vs maven-assembly-plugin

I wrote a hadoop MR job using spring-data-hadoop and used maven-shade-plugin to pack all necessary jars. But when I ran the job from comand line:hadoop jar , it gave me Exception for ClassNotFound for the spring-beans. Resulting problem in loading…
user1516083
  • 81
  • 7
3
votes
0 answers

Spring for Hadoop HBase

I have a question to Spring community. As stated in the article, Spring for Hadoop will no longer be supported on April, 2019. Is (or will) there be any other Spring project for Hadoop, HBase (especially new versions 2.x.x)?
Markiza
  • 444
  • 1
  • 5
  • 18
3
votes
1 answer

trap Spring context close error

I have a spring Integration project using spring-hadoop. I have an Hbase template as follows,
ameet chaubal
  • 1,440
  • 16
  • 37
3
votes
1 answer

Spring Boot Yarn - Passing Command line arguments

i'm trying to pass command line arguments in my Spring Boot Yarn application and am having difficulties. i understand that i can set these in the yml document spring.yarn.appmaster.launchcontext.arguments but how can it from the command line? like…
incomplete-co.de
  • 2,137
  • 18
  • 23
2
votes
1 answer

Spring Hadoop config - No qualifying bean of type org.apache.hadoop.conf.Configuration

I am trying to configure beans for Hadoop/Hive environment. According to documentation I need Apache Hadoop Configuration class, which should be autowired. See:…
wst
  • 4,040
  • 4
  • 41
  • 59
2
votes
1 answer

Unknown host exception when using spring data hadoop to connect to Cloudera QuickStart VM Hbase

I use QuickStart VMs for CDH 5.3.x I try to implement this spring hadoop sample for hbase The sample from the host computer will connect to Hbase in the VM to create table, add data, read data. In my pom i use …
SieuCau
  • 195
  • 1
  • 2
  • 15
2
votes
2 answers

spring data hive integration with hive template

I am trying to use spring data hadoop to integrate hive into my application and running into some issues. First thing I am not sure about is is this to connect to an existing hive server or to…
adeelmahmood
  • 2,371
  • 8
  • 36
  • 59
1
vote
1 answer

Spring HbaseTemplate keeping connection alive

I managed to integrate Hbase into a Spring app using HbaseTemplate: import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.hadoop.hbase.HbaseTemplate; import org.springframework.stereotype.Component; import…
pawello2222
  • 46,897
  • 22
  • 145
  • 209
1
vote
1 answer

How to "findAllBy{Column}" with HbaseTemplate?

I have been developing my first RESTful server utilizing such technologies as Spring and HBase. Message below is the core model of my server; @AllArgsConstructor @Getter public class Message { private final String from; private final…
Pasha
  • 1,768
  • 6
  • 22
  • 43
1
vote
1 answer

Spring Hadoop (Hive)

Am I supposed to understand how to implement Spring Hadoop / Hive by reading this quick guide? I don't know where the xml configurations should go (maybe…
Geoff L.
  • 99
  • 1
  • 2
  • 9
1
2 3 4