Rails save log data to database

Question

Is it possible to access to the information being saved into a rails log file without reading the log file. To be clear I do not want to send the log file as a batch process but rather every event that is written into the log file I want to also send as a background job to a separate database.

I have multiple apps running in docker containers and wish to save the log entries of each into a shared telemetry database running on the server. Currently the logs are formatted with lograge but I have not figured out how to access this information directly and send it to a background job to be processed.(as stated before I would like direct access to the data being written to the log and send that via a background job)

I am aware of the command Rails.logger.instance_variable_get(:@logger) however what I am looking for is the actual data being saved to the logs so I can ship it to a database.

The reasoning behind this is that there are multiple rails api's running in docker containers. I have an after action set up to run a background job that I hoped would send just the individual log entry but this is where I am stuck. Sizing isn't an issue as the data stored in this database to be purged every 2 weeks. This is moreso a tool for the in-house devs to track telemetry through a dashboard. I appreciate you taking the time to respond

Possible duplicate of [Log to database instead of log files](https://stackoverflow.com/questions/1160720/log-to-database-instead-of-log-files) — dbugger, Oct 31 '18 at 16:52

score 0 · Answer 1 · answered Oct 31 '18 at 15:28

You would probably have to go through your app code and manually save the output from the logger into a table/field in your database inline. Theoretically, any data that ends up in your log should be accessible from within your app.

Depending on what how much data you're planning on saving this may not be the best idea as it has the potential to grow your database extremely quickly (it's not uncommon for apps to create GBs worth of logs in a single day).

You could write a background job that opens the log files, searches for data, and saves it to your database, but the configuration required for that will depend largely on your hosting setup.

@cassidb7 Cool. If you wouldn't mind adding that clarification to your question I'll delete this answer. — NM Pennypacker, Oct 31 '18 at 15:57

score 0 · Answer 2 · answered Nov 02 '18 at 09:00

So I got a solution working and in fairness it wasn't as difficult as I had thought. As I was using the lograge gem for formatting the logs I created a custom formatter through the guide in this Link.

As I wanted the Son format I just copied this format but was able to put in the call for a background job at this point and also cleanse some data I did not want.

module Lograge
  module Formatters
    class SomeService < Lograge::Formatters::Json
      def call(data)
        data = data.delete_if do |k|
          [:format, :view, :db].include? k
        end

        ::JSON.dump(data)

        # faktory job to ship data
        LogSenderJob.perform_async(data)
        super
      end
    end
  end
end

This was just one solution to the problem that was made easier as I was able to get the data formatted via lograge but another solution was to create a custom logger and in there I could tell it to write to a database if necessary.

Rails save log data to database

2 Answers2