0

What is standard(industry standard) way to keep a Java program running continuously.

Use case(Realtime processing):

  • A continuous running Kafka producer
  • A continuous running Kafka consumer
  • A continuous running service to process a stream of objects

Found few questions in Stackoverflow, for example: https://stackoverflow.com/a/29930409/2653389

But my question is specific to what is the industry standard to achieve this .

J-Alex
  • 6,881
  • 10
  • 46
  • 64
Pintu
  • 112
  • 9
  • 2
    There is no industry standard. There might be several proven solutions / design patterns but it depends so much on the specific use case what is the best one. – Joakim Danielson Apr 26 '18 at 08:19
  • I have shared few use cases, especificially to process realtime stream of data – Pintu Apr 26 '18 at 08:21

2 Answers2

3

First of all, there is no specified standard.

Possible options:

  • Java EE WEB application
  • Spring WEB application
  • Application with Spring-kafka (@KafkaListener)

Kafka producer will potentially accept some commands. In real-life scenario I worked with applications which runs continuously with listeners, receiving requests they triggered some jobs, batches and etc.

It could be achieved using, for example:

  • Web-server accepting HTTP requests
  • Standalone Spring application with @KafkaListener

Consumer could be a spring application with @KafkaListener.

@KafkaListener(topics = "${some.topic}")
public void accept(Message message) {
   // process
}

Spring application with @KafkaListener will run infinitely by default. The listener containers created for @KafkaListener annotations are registered with an infrastructure bean of type KafkaListenerEndpointRegistry. This bean manages the containers' lifecycles; it will auto-start any containers that have autoStartup set to true. KafkaMessageListenerContainer uses TaskExecutor for performing main KafkaConsumer loop.

Documentation for more information.

If you decide to go without any frameworks and application servers, the possible solution is to create listener in separate thread:

public class ConsumerListener implements Runnable {

    private final Consumer<String, String> consumer = new KafkaConsumer<>(properties);

    @Override
    public void run() {
        try {
            consumer.subscribe(topics);

            while (true) {
                    // consume
                }
            }
        } finally {
            consumer.close();
        }
    }
}
J-Alex
  • 6,881
  • 10
  • 46
  • 64
  • Regarding the statement `Application with @KafkaListener will run infinitely.` Does kafka take care of this ? – Pintu Apr 26 '18 at 08:52
  • @Pintu, see the updated answer. Actually this is the feature of spring-kafka project. – J-Alex Apr 26 '18 at 09:26
  • got it. If i have to go without Spring, we need a infinite while loop to keep the listener running , correct ? – Pintu Apr 26 '18 at 09:43
  • @Pintu yes, if you will decide to go withour application servers/web containers. One more update for the answer. – J-Alex Apr 26 '18 at 10:22
1

When you start your program like "java jar" it will work until you didn't stop it. But that is OK for simple personal usage and testing of your code. Also in UNIX system exist the app called "screen", you can run you java jar as a daemon.

The industry standard is application servers. From simple Jetty to enterprise WebSphere or Wildfly(ex. JBoss). The application servers allows you to run application continiously, communicate with front-end if neccassary and so on.

Danila Zharenkov
  • 1,720
  • 1
  • 15
  • 27
  • Yes I can host the service on App server, but would be useful for on-demand processing. I am looking for a solution for real-time processing. Edited question with specific use case. – Pintu Apr 26 '18 at 08:25
  • 2
    The EE standard is application servers, pretty much everybody outside of EE would now deploy a container (as in e.g. docker container and not a war to make it clear) running `java -jar` internally – Oleg Sklyar Apr 26 '18 at 08:27
  • @Pintu your test cases aren't about running app, but about patterns. Take a look at Publish/subscribe pattern, producer/consumer pattern. I'm not a Kafka expert, but as I know it uses some kind of queues(topics), which are used to publish and consume data by producer and consumer respectively – Danila Zharenkov Apr 26 '18 at 08:31
  • @Danila My question in not realated I hosting application or running java command or about pattern. Rather what implementation logic will let process run and do processing continuosly. The link I have shared has `while` loop and `wait ()` as solution .Is it standard way or any other way to achieve it. – Pintu Apr 26 '18 at 08:42
  • Mesos / Kubernetes / Docker Swarm aren't Industry standards yet? – OneCricketeer Apr 29 '18 at 00:40