Stack Overflow
Stack Overflow

Questions tagged [producer]

Producer is an actor in the Producer Consumer scenario

Producer is an actor in the Producer Consumer scenario. The producer is the one who produces the data to be consumed by the consumers. See

237 questions
246
votes
27 answers

Leader Not Available Kafka in Console Producer

I am trying to use Kafka. All configurations are done properly but when I try to produce message from console I keep getting the following error WARN Error while fetching metadata with correlation id 39 : {4-3-16-topic1=LEADER_NOT_AVAILABLE}…
Vishesh
  • 3,599
  • 4
  • 22
  • 36
46
votes
1 answer

Fast and Best Producer/consumer queue technique BlockingCollection vs concurrent Queue

Im using Generic.Queue in C# 3.0 and Monitor.Enter,wait,exit for wait before consuming the queue (wait for the element to be enqueued). Now im moving to C# 4. Can anyone suggest me which one is fast and best especially to avoid…
C-va
  • 2,910
  • 4
  • 27
  • 42
21
votes
5 answers

Spring Kafka asynchronous send calls block

I'm using Spring-Kafka version 1.2.1 and, when the Kafka server is down/unreachable, the asynchronous send calls block for a time. It seems to be the TCP timeout. The code is something like this: ListenableFuture> future =…
Carlos E. L. Augusto
  • 223
  • 1
  • 2
  • 7
21
votes
5 answers

Interact with kafka docker container from outside of docker host

I have built a kafka docker container and orchestrate it using docker-compose. Calling docker ps I get the following putput: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS …
Trung Hiếu Trần
  • 365
  • 1
  • 2
  • 10
20
votes
3 answers

Understanding the max.inflight property of kafka producer

I work on a bench of my Kafka cluster in version 1.0.0-cp1. In part of my bench who focus on the max throughput possible with ordering guarantee and no data loss (a topic with only one partition), need I to set the…
Quentin Geff
  • 819
  • 1
  • 6
  • 21
16
votes
2 answers

What is the Spring DI equivalent of CDI's InjectionPoint?

I would like to create a Spring's bean producer method which is aware who invoked it, so I've started with the following code: @Configuration public class LoggerProvider { @Bean @Scope("prototype") public Logger produceLogger() { …
Piotr Nowicki
  • 17,914
  • 8
  • 63
  • 82
14
votes
3 answers

Topic can't be found when producing messages: UNKNOWN_TOPIC_OR_PARTITION

I have a two-nodes Kafka cluster (EC2 instances) where each node is used as a separate broker. When I run a producer on the leader instance with the following command: kafka-console-producer.sh --broker-list localhost:9092 --topic test I get the…
saloua
  • 2,433
  • 4
  • 27
  • 37
8
votes
4 answers

Producer/consumer system using database (MySql), is this feasible?

I need to use something to coordinate my system with several consumers/producers each running on different machines with different operating systems. I have been researching on using MySql to do this, but it seems ridiculously difficult. My…
johnrl
  • 873
  • 3
  • 14
  • 18
7
votes
1 answer

RabbitMQ - Consume multiple queues by priority

I have a requirement to upgrade our Producer/Consumer infrastructure. The current setup look like this: Set of 3 queues with different priority (low, med, high). When one our customers generate a task (i.e Process an image): The Producer add the…
eldad87
  • 195
  • 1
  • 13
6
votes
4 answers

How to upload the data from python sdk to kinesis using boto3

How to upload the data from csv to aws kinesis using boto3 I have tried three methods and it is all working for me. To upload the data from csv to kinesis in chunks. Upload the random generated data from local to kinesis. Upload the csv data row by…
Saurabh Kukreti
  • 119
  • 1
  • 6
6
votes
2 answers

oData4j sample eclipse project

I have downlaoded oData4J jar files from odata.org and I intend to do a sample producer, within a basic web service. I assumed that InMemoryProducerExample.java from http://code.google.com/p/odata4j/ will do the job and most probably it is. My…
Iftemi Alin
  • 485
  • 4
  • 14
5
votes
2 answers

Best way to implement a multiple producer/consumer pattern in Java 6

So I have multiple steps stage 1 -> stage 2 -> stage 3 -> stage4 so in some cases the producer would be a consumer, and at each stage there are multiple producers/consumers to make use of multiple cpus. In case relevent some packets would miss out…
Paul Taylor
  • 13,411
  • 42
  • 184
  • 351
5
votes
1 answer

RabbitMQ with Arduino Uno

I'm using RabbitMQ with Arduino for the first time and I need to publish data. So I've used the PubSubCLient class. This is the code: #include #include #include #include #include…
Anas Elazhar
  • 112
  • 1
  • 12
5
votes
2 answers

Kafka Producer Exception NoClassDefFoundError

I have some problem with kafka Producer, but i dont know how i can solved it My Maven Dependency: org.apache.kafka kafka_2.10 0.10.1.1 If i…
ofitz
  • 152
  • 1
  • 2
  • 14
5
votes
1 answer

Reliable fire-n-forget Kafka producer implementation strategy

I'm in middle of a 1st mile problem with Kafka. Everybody deals with partitioning, etc. but how to handle the 1st mile? My system consists of many applications producing events distributed on nodes. I need to deliver these events to a set of…
Yuri
  • 4,254
  • 1
  • 29
  • 46
1
2 3
…
15 16