1

I'm running a distributed job on multiple Linux nodes. Each node logs data into one file. I lose a lot of time using ssh to connect and view each log individually.

Is there a tool to consolidate file from across systems? or another way to view multiple logs at once?

Matthias J. Sax
  • 59,682
  • 7
  • 117
  • 137
cldo
  • 1,735
  • 6
  • 21
  • 26

1 Answers1

4

I would recommend looking at Splunk, it's a wonderful tool for managing log files across many systems, I've been using it for a while in large Hadoop systems and it will basically aggregate data in real-time from all your logs and give you a holistic view of your system in a UI so you don't have to log onto every system and you can see everything in the UI.

It also lets you define alerts, and you can filter everything and look at individual hosts if you want.

Charles Menguy
  • 40,830
  • 17
  • 95
  • 117
  • I agree that Splunk is probably the best option if you can afford it. For other options see http://stackoverflow.com/questions/83741/best-way-to-aggregate-multiple-log-files-from-several-servers – Gordon Seidoh Worley May 31 '13 at 14:41