0

I know docker service logs get logs from containers which are part of that service. But how will it fetch? is it fetch once and cache somewhere or every time I issue command "docker service logs" it will fetch the logs via network?

Ajay Naredla
  • 408
  • 5
  • 17
  • Docker does cache the logs because it gathers the `stdout` and `stderr` prints. But it will cache the logs on the docker engine where service is running. So if you access the logs from the same machine where service is running, obviously you won't go over the network. But in case you access logs from a manager and the service is deployed on some worker node, logs will be transported over the network. In either case, you'll get cached logs. – Saqib Ahmed Apr 24 '18 at 07:07
  • tq @SaqibAhmed.. But where they will be cached on the manager, I want to know the exact location – Ajay Naredla Apr 24 '18 at 07:40
  • Services in the docker are actually containers and their logs are placed `/var/lib/docker/containers//-json.log` as the answer suggests. – Saqib Ahmed Apr 24 '18 at 07:54
  • Try using `docker service ps ` to see the machines where services are running in containers. Simply go to the machine you're interested in and find the container id with `docker ps`. You can then see the logs in its respective directory. Hope it clarifies. – Saqib Ahmed Apr 24 '18 at 07:56
  • tq @SaqibAhmed. I know that information. what I want to know is --"On manager machine where are the logs for a service will be cached(If cached)?" – Ajay Naredla Apr 24 '18 at 08:07
  • My problem is "docker logs " logs are good for a conrtainer, but docker service logs is giving me the error " received message length 1869051448 exceeding the max size 4194304 " – Ajay Naredla Apr 24 '18 at 08:08
  • Oh. OK. Now I got your question. As I said in my first comment if you're accessing logs from manager and container's running on a worker (or some other node) these logs will be cached there but they won't be cached on the manager. They will be cached on their corresponding node, so the manager will always have to pull them from that machine over the network. – Saqib Ahmed Apr 24 '18 at 08:09
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/169648/discussion-between-ajay-naredla-and-saqib-ahmed). – Ajay Naredla Apr 24 '18 at 08:11

2 Answers2

1

As mentioned in my comment and the other answer, docker engine always caches logs of the containers running on that docker engine and store them in /var/lib/docker/containers/<container id>/<container id>-json.log directory. When you do docker service logs from a machine where the containers of the said service are not running, docker always pulls the log from the machine over the network and it never caches.

That being said, the error you're facing received message length 1869051448 exceeding the max size 4194304 is because there might be a log line that is simply too long to fit in the gRPC object being sent across the network.

Solution

  1. Specify the --tail <n> option to docker service logs where n is the number of lines from the end of the logs you want to see
  2. Specify a task ID from docker service ps instead of a service name, giving you the logs from that task alone rather than the aggregated logs from across the service replicas.

This might still give you the error if you still have that long log line in your pulled logs.

Saqib Ahmed
  • 1,056
  • 14
  • 33
  • Thank you very much for your effort.. because of one line being unable to fit in gRPC object I can't able get any log further, now tailing helps alot.. – Ajay Naredla Apr 24 '18 at 09:17
0

By default docker logs to: /var/lib/docker/containers/<container id>/<container id>-json.log

This question is already answered.

For some advanced logging options see logging drivers.

Ádám Révész
  • 231
  • 2
  • 8
  • I am talking about docker service logs .. not docker container logs.. So where will they get cached in the docker manager? – Ajay Naredla Apr 24 '18 at 07:22
  • Oh, excuse me. In that case I see [Saqib Ahmed's comment](https://stackoverflow.com/questions/49995323/how-will-docker-service-logs-fetch-logs/49995403?noredirect=1#49995323) under your question. :) – Ádám Révész Apr 24 '18 at 07:27
  • yup. He is saying logs will be cached but my question is where they will be stored ? I mean path – Ajay Naredla Apr 24 '18 at 07:30