How to collect access logs from different nodes?

Question

We have a lot of nodes (about 18) on Production, and it is necessary to find out if the specific user made requests from the mobile app (access logs have this info).

SAP Hybris doesn't store such info in the database, so it is necessary to search the logs. Each node has a file like log/tomcat/access.2018-11-22.log. It would be unreasonable to check each file manually.

Maybe there is a well-known solution for the situations like this?

This might also be helpful: https://stackoverflow.com/questions/83741/best-way-to-aggregate-multiple-log-files-from-several-servers — John Camerin, Nov 22 '18 at 14:10

score 2 · Answer 1 · answered Nov 22 '18 at 14:06

2

There are 3rd party solutions such as Splunk and ELK which will collect the data from log files.

answered Nov 22 '18 at 14:06

John Camerin

677
5
15

score 1 · Answer 2 · answered Nov 22 '18 at 17:23

1

Take a look on it

Its a ELK stack, very common and easy to use in production

answered Nov 22 '18 at 17:23

Ihar Sadounikau

741
6
20

score 0 · Answer 3 · answered Nov 22 '18 at 18:10

0

If you want to use a tool then i would recommend to use Splunk else you can log request in database as per the requirements.

If you can share the requirements in detail then i am happy to help you providing better solution.

answered Nov 22 '18 at 18:10

user3624540

56
4

score 0 · Answer 4 · answered Nov 23 '18 at 15:03

0

We are developing micro service projects and have about 20 services. All the services connect to ELK(Elastic search,Logstash,Kibana). We are viewing all the log though Kibana.It is very easy make queries to search what we want.

answered Nov 23 '18 at 15:03

WGSSAMINTHA

180
1
11

score 0 · Answer 5 · answered Nov 25 '18 at 01:37

0

If you're interested, Hybris Help also has a guide on how to setup B2C Accelerator in a Docker Container with Centralized Logging: https://help.hybris.com/1808/hcd/af36215ea1e34c22a1e3cbb2d550f733.html

answered Nov 25 '18 at 01:37

geffchang

3,279
2
32
58

score 0 · Answer 6 · answered Nov 26 '18 at 16:04

Use ELK - elasticsearch logstash and kibana stack with filebeat. Filebeat -will share the log file content with logstash Logstash-will scan, filter and share the needed content to elastic search Elasticsearch- will work as a db, store the content from logstash in json format as documents. Kibana- with kibana you can search the required info. Also you can plot graphs and other visuals with the relevant data.

How to collect access logs from different nodes?

6 Answers6