2

The goal of this context is to achieve a centralized logs storage. I want all of the logs across different projects to be stored under the same project.

By far of my experience on the ops-agent, it always sends logs to the corresponding project's log bucket. The ops-agent documentation neither provide manual on customizing the log destination.

I want to send the logs to a specified log bucket or Pub/Sub topic. Is it achievable?

kristen_1024
  • 33
  • 5

1 Answers1

2

I think that Log Sinks could help you to achieve what you want. They are the mecanism to aggregate logs within and Organization.

Here you can check and example where they do the same with audit logs.

And here you can find the documentation of how to route logs using sinks.

With sinks you can use several destinations as Log buckets, PubSub, cloud Storage or BigQuery.

Hope this works for your use case.

Oscar Ferrer
  • 126
  • 1
  • 4
  • So, there is no way to directly configure ops-agent to point to the destination bucket or Pub/Sub topic I want. Instead, I can create a Log Sink rule at organization or folder level so that it will apply to all of the projects underneath. Then, every log once arrive the default bucket, it will sink to my centralized destination according to the Log Sink rule. Finally, my centralized logging is achieved. Is that true? – kristen_1024 Aug 17 '22 at 15:10
  • The recommended way to do it is as you pointed out -> create log sink rules at the level you wish to aggregate logs from your project. You can use also more granular filters as for example, only sink logs from VMs with label "env:pro". – Oscar Ferrer Aug 18 '22 at 11:40
  • I have just tested another option that is to configure in a VMs a service account (SA) from other project. It worked. That is: the logs from a VM in project A goes to logs bucket in project B without doing anything more than setting up de SA. Although the docs says that you should use a SA from the same project of your VM you can use a SA from the project where you want to store the logs https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/authorization In any case, this options seems more like a workaround... the Sink option is cleaner and more maintainable. – Oscar Ferrer Aug 18 '22 at 11:52
  • How do you configure the VM so that the ops-agent sends logs to another project's bucket? Could you explain this part a little bit? – kristen_1024 Aug 18 '22 at 13:08
  • 1
    Basically you can following this steps to create a service account and uploading the key to the VM --> https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/authorization#private_key_authorization. The only difference is that the service account should be created in the google project where you are going to store the logs. But as as said this has other considerations and I would not recommend it. – Oscar Ferrer Aug 30 '22 at 14:35