1

We are developing a service for our QA staff.

The main goal is that a tester from our web interface be able to select from a github branch a dump for this particular machine and click "Deploy" button, then the rails app for testing will be deployed to Digital Ocean.

The feature I am now working on, is collecting deployment logs and displaying them through our web interface.

On DO droplet there is a "logs" folder which contains different log files which are populated during deployment:

migrations_result_#{machine_id}.log, bundle_result_#{machine_id}.log, etc.

Where #{machine_id} is the id of deployed machine on our service(it is not droplet id).

With the help of remote_syslog gem we are monitoring "logs" folders on each droplet and send them through udp to our main service server, and with the help of rsyslog we store them in a particular folder, let's say /var/log/deplogs/

So in /var/log/deplogs/ we have:

migrations_result_1.log, bundle_result_1.log,

migrations_result_2.log, bundle_result_2.log,
...

migrations_result_n.log, bundle_result_n.log

How do I need to monitor this folder and save contents of each log file to mysql database?

I need to achieve something like the following (Ruby code):

Machine.find(#{machine_id}).logs.create!(text: "migrations_result_#{machine_id}.log contents")

Rsyslog does not seems to be able to achieve this. Or am I missing something? Any advices?

Thanks in advance, and sorry for my English, I hope you can get the idea.

Ramon Araujo
  • 1,743
  • 22
  • 31
user1136228
  • 967
  • 9
  • 22

1 Answers1

0

First of all, congratulations! You are in front of a beautiful problem. My suggestion is to use divide and conquer.

Here are my considerations:

  • Put the relevant folder(s) under version control (for example, GIT)
  • Check via GIT commands the files that changed every X amount of time.
  • Also obtain the differences between the prior version of each file, and the new ones, so you can update your database parsing the new info.

Just in case, here are ways to call system commands from ruby.

Hope that helps,

Community
  • 1
  • 1
Ramon Araujo
  • 1,743
  • 22
  • 31