How to generate input data and stored it in to a local file and How we can read the data of this input generated file by using kafka

Question

For this i started Zookeeper: bin/zookeeper-server-start.sh config/zookeeper.properties

then the kafka server: bin/kafka-server-start.sh config/server.properties

topic: bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic fast-messages

then what is the next step for file generation ?

Answer 1

You need to parse the file, and write a Kafka producer from it.

Or you can just pipe the file directly into the Console Producer.

How to write a file to Kafka Producer

Or use KafkaConnect FileSourceConnector

Answer 2

For this same Question i also referred another link and i got the result. The Link is --> https://docs.confluent.io/current/streams/quickstart.html

Followings are the step that i followed-->

Step 1: Start Zookeeper bin/zookeeper-server-start.sh config/zookeeper.properties

Step 2: Start kafka server bin/kafka-server-start.sh config/server.properties

Step 3: create a topic named as "fast-messages" bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic fast-messages

step 4: Run Producer kafka-console-producer.sh --broker-list localhost:9092 --topic fast-messages --new-producer < my_file.txt

step 5: Listened by the Consumer:

/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic fast-messages --from-beginning