1

This question is for asking general advice on what tools should I use for this task, and possibly pointing me to some related tutorials.

I have a Spring Boot web application, which during operation generates log messages into a database. There is a JavaScript management tool in the making for this REST application, and 1 function of it would be to show the log messages real-time. Meaning, when the user is on the log showing page, he should see the new log messages appearing without refresing the page.

My questions:

  • What should be used to provide this for the javascript client at some endpoint? I'm looking at these spring boot starters right now: websocket, redis, amqp. I have not used any of these before.
  • How should I "catch" the log messages as they are generated inside the application? So I could send them to the client with the chosen solution.

I'm not really looking for a periodic query type of solution, but rather a server pushing the data as it appears solution.

Any suggestions and code samples are appreciated.

logi0517
  • 813
  • 1
  • 13
  • 32
  • Websockets looks good for what you are trying to achieve. Since this is linked to logs, maybe you can check elastic search's plugins for Websocket output. So you don't have to rewrite everything yourself. – Akli REGUIG Jul 20 '18 at 08:54

1 Answers1

3

Storing logs in a database is usually not a good option unless you use a database which is capable of handling a lot of write requests, such as Apache Cassandra. Streaming data from a database is not the most intuitive thing to do, however.

A modern alternative is to use a messaging system such as Apache Kafka to stream logs from producing systems to multiple subscribing systems. There are multiple ways how you can achieve that. For example, for streaming logs from your Spring Boot app you could use a special log4j appender (see here and an example here). To be able to present logs in a web browser in real-time, you will need another backend system which will receive the log records from Kafka topics and forward them to JavaScript web clients via websockets, most likely using a publisher/subscriber model.

Also, you could consider using server sent events (SSE) instead of websockets. Because you have only a unidirected message flow (logs are streamed from a backend system to a javascript client in the browser, but not the other way around), SSE can be a good option as a replacement for websockets. Websockets are more difficult to operate than SSE and usually use more resources at the backend. As always, you will need to choose between trade-offs (see here).

AlexLiesenfeld
  • 2,872
  • 7
  • 36
  • 57
  • you misunderstood my post a bit. the need for this functionality just came up recently, I havent implemented anything yet to achieve this. I only have the REST parts, which create the logs at the moment. I mentioned those 3 things as a potential way to achieve these maybe, but I didn't use them before, that is why i was asking for advice. SSE-s are a good idea though, thanks for reminding me about those. my other big question mark is how should I intercept the log messages. – logi0517 Jul 20 '18 at 09:55
  • I see. I added the idea of using Apache Kafka for streaming logs. – AlexLiesenfeld Jul 20 '18 at 12:01