0

A have a project which regularly performs a certain task. During the processing of this task I'm logging information to a log file using Spring-Boot. The task itself that runs is run via an Async method.

I've used a logback.xml file to ave logs for this particular class to it's own log file.

What I'd like to do it have a web page, which makes a call to a an endpoint in the spring boot application which in turn returns the contents of the log file. The web page should automatically update the log file data displayed to the user.

I've tried to create this, and I've had some success (it works until the log file reaches a particular size), however the log file is now over 15MB and I'm struggling to have the file auto-updated in a web browser every 2 seconds as it's taking to long to load.

The steps I'm taking currently are; - Starting the process, this returns the number of lines currently in the log file (this is so that the web page can request lines from the log file after this point so that it doesn't ask for historical logs) - A web page is loaded in the browser, this make a call to the API every 2 seconds asking for all lines of the log file after the number returned in the first call. - The returned log file data is populated into a div, and jquery is used to scroll to the bottom of the div so the user can see the latest entries in the log file.

Currently the log file has 192,000 lines in it, I'm assuming the issue is loading this into memory, then counting the number of lines in the file. Then each time the API is called, we're loading the whole file again, looping through each line and only returning lines after the original number of lines returned by the API initially.

Loading the contents of the log file like this doesn't seem to be the solution I'm looking for.

Can anyone suggest an alternative?

SheppardDigital
  • 3,165
  • 8
  • 44
  • 74
  • 1
    It seems like you already have a good handle on the problem - Loading the entire file for every request is not practical. Instead, you need to do something similar to [`tail`](http://man7.org/linux/man-pages/man1/tail.1.html). How you implement that is up to you, but [here is somewhere to start](http://stackoverflow.com/questions/4121678/java-read-last-n-lines-of-a-huge-file). – nickb Apr 13 '17 at 14:37
  • 1
    have a look at the actuators dependency in Spring Boot http://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html, in particular the logfile endpoint. Also, consider sending your logs to elastic search and create filters in kibana to visualise your log files (and make sense of them) – Eduardo Sanchez-Ros Apr 13 '17 at 14:50

0 Answers0