Stack Overflow
Stack Overflow

Questions tagged [spring-xd]

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Spring XD is a unified, distributed, and extensible system for data ingestion, real time analytics, batch processing, and data export. The project’s goal is to simplify the development of big data applications.

Big data applications share many characteristics with Enterprise Integration and Batch applications. Spring has provided proven solutions for building integration and batch applications for more than 6 years now via the Spring Integration and Spring Batch projects. Spring XD builds upon this foundation and provides a lightweight runtime environment that is easily configured and assembled via a simple DSL.

Click here for more information about Spring XD.

565 questions
24
votes
1 answer

unable to deploy spring-xd processor module using spring-data-jpa on xd-singlenode

I have a spring-xd processor module which has a dependecy to a project using spring-data-jpa: pom.xml of the xd-processor module: org.test.common …
Domi
  • 1,451
  • 2
  • 13
  • 25
10
votes
3 answers

Spring Batch - Read files from Aws S3

I am trying to read files from AWS S3 and process it with Spring Batch: Can a Spring Itemreader process this Task? If so, How do I pass the credentials to S3 client and config my spring xml to read a file or multiple files
sve
  • 393
  • 1
  • 2
  • 15
7
votes
3 answers

how to stream data out of a cassandra table?

I would like to stream data from a cassandra table which is updated in real time. Yes, it is a database but is there a way to do that? If so, keeping an offset or what CQL queries can I use ?
krish at
  • 89
  • 1
  • 4
7
votes
3 answers

How to run read() function only once Spring Batch

I am creating a Spring Batch job with folowing xml:
Amaresh
  • 3,231
  • 7
  • 37
  • 60
7
votes
2 answers

Apache Spark streaming vs Spring XD Streams

I am looking for use cases where Apache Spark streaming will be apt and situations where Spring XD streams will best fit. Is there a overlap of technologies here? Has someone come across a situation where both are being used?
Rishitesh Mishra
  • 459
  • 5
  • 9
6
votes
1 answer

spring integration vs reactive-streams

I'm working on an ETL project. I've been using spring integration for a long time. The data source is currently files or chronicle but it may change to live streams and volumes are likely to grow. There is a potential to move on to big data…
fitifiti
  • 97
  • 6
6
votes
1 answer

How to get Spring XD to deploy a predefined set of streams and taps on startup

Is there a way to launch a set of steams and taps that have been defined when a new instance of spring xd is launched? The scenario is this: We've done some prototyping on our dev systems, but now when we'd like to build and release this to a test…
Arun Jose
  • 365
  • 4
  • 16
6
votes
2 answers

"Spring xd" versus storm

I'm actively looking for a real time big data analysis solution. for now I've identified a mature actor Twitter Storm and also a young challenger Spring XD. Does anyone know if Spring XD could real time process log event streams or twitter streams…
tom
  • 1,647
  • 14
  • 15
5
votes
1 answer

Spring Cloud data flow documentation

Any update on when the Spring Cloud Data Flow documentation will be ready for prime time. It is really hard to plan for any significant deployment in the future, without complete developer documentation. Since Spring Cloud Data Flow draws off Spring…
Vinay Joshi
  • 51
  • 1
5
votes
2 answers

Spring-XD does not read logback.xml

I am trying to run a job in Spring-XD, located under the following path: /spring-xd/xd/modules/job/MyJobName (I'll call this path MyJobName below) My jar, located under MyJobName/lib, contains in its root path the file logback.xml. Unfortunately,…
Jonathan Pitre
  • 2,355
  • 3
  • 22
  • 38
4
votes
0 answers

Class org.apache.kafka.common.serialization.StringSerializer not found

I wrote a Kafka producer that runs well when I execute it in anyplace besides Spring-XD daemon. private Properties setProperties () { Properties props = new Properties(); …
Mantovani
  • 500
  • 2
  • 7
  • 18
4
votes
1 answer

Kafka Source in Spring Cloud Data Flow

I am migrating from Spring XD to Spring Cloud Data Flow. When I am looking for module list I realised that some of the sources are not listed in Spring Cloud Flow - One of them is KAFKA source. My question is why KAFKA source is removed from…
Viren Balaut
  • 55
  • 2
  • 6
4
votes
1 answer

Does Spring Cloud data flow support batch processing

We are exploring various programming/library options (on Java side of the world) for faster batch processing as well as be able to be deployed on cloud. We came across Spring batch/XD/cloud data flow. From the quick review of documentation on…
JavaTec
  • 965
  • 3
  • 15
  • 30
4
votes
1 answer

sending TCP message via command line on macosx

For testing purposes I am sending tcp messages to a local server as follows: echo -e "some message" | netcat localhost 1234 With netcat installed using brew install netcat. This works fine except for that this blocks for quite a long time (about a…
user152468
  • 3,202
  • 6
  • 27
  • 57
4
votes
2 answers

Spring Integration Kafka adaptor not producing message

I am struggling this for days now. I am using SI adaptor for kafka under Spring-boot container. I have configured zookeeper and kafka on my machine. I also created console producer and consumer tested it and everything works fine(I manage to…
rayman
  • 20,786
  • 45
  • 148
  • 246
1
2 3
…
37 38