19

I'm looking to set up some alerts from gcloud -> slack, and so far have a test up and running having followed these instructions:

https://cloud.google.com/monitoring/support/notification-options?_ga=2.190773474.-879257953.1550134526#slack

However, ideally I'd store the config for these notifications in a terraform script so that I don't have manual steps to follow if things need setting up again. It looks like this should be possible: https://www.terraform.io/docs/providers/google/r/monitoring_notification_channel.html

I've run gcloud alpha monitoring channel-descriptors describe projects/<My Project>/notificationChannelDescriptors/slack, which produces the following output for the labels+type:

labels:
- description: A permanent authentication token provided by Slack. This field is obfuscated
    by returning only a few characters of the key when fetched.
  key: auth_token
- description: The Slack channel to which to post notifications.
  key: channel_name
type: slack

So, I think my terraform config for the notification channel wants to be:

resource "google_monitoring_notification_channel" "basic" {
  display_name = "My slack notifications"
  type = "slack"
  labels = {
    auth_token = "????????"
    channel_name = "#notification-channel"
  }
}

However, I can't figure out how to obtain the auth token for this script? I can't seem to extract the one I've already set up from Slack or gcloud, and can't find any instructions for creating one from scratch...

N.B. This is not a Terraform-specific issue, because the script is just hooking into the google REST API. So, anyone using the API directly would also have to obtain this auth_token from somewhere. There must be an intended way to obtain it or why is it in the API at all...?

Alyssa
  • 835
  • 1
  • 7
  • 23

2 Answers2

14
  1. Visit https://app.google.stackdriver.com/settings/accounts/notifications/slack?project=YOUR_PROJECT_NAME
  2. Select "Add Slack Channel"
  3. Select "Authorize Stackdriver"
  4. Select "Install"
  5. You will be redirected back to a URL of the form: https://app.google.stackdriver.com/settings/accounts/notifications/slack/add?project=YOUR_PROJECT_NAME&auth_token=AUTH_TOKEN_HERE
  6. Save the notification channel (this seems to be necessary to finish the oauth flow)
  7. Copy/paste the auth token from the &auth_token= parameter in the query string

You will end up with an extra notification channel, i.e. the one you created in the console, but after that you will be able to reuse the auth token in terraform-managed notification channels.

Matt Zimmerman
  • 529
  • 5
  • 10
  • 1
    The URL which contains the auth token is no longer the final URL that we are redirected to. To find the auth token, open the browser's network monitor and follow the above steps. In the list of requests made throughout the process you will find that one of them includes the auth token – noamt May 11 '20 at 17:37
  • You can delete the extra notification channel afterwards and the token will continue to work. – dshepherd Jun 11 '20 at 10:22
  • 1
    This doesn't work anymore since stackdriver was merged into monitoring. :( – maroux Sep 24 '20 at 21:57
  • Is it actually safe to copy and paste this in terraform ? Wondering if i should avoid provisionning the channel if i dont have any way to hide this token – Rose Dec 11 '20 at 02:45
  • 1
    @Rose you'd want to throw the token into a GCP Secret and reference it here. Like other sensitive values in terraform, it shouldn't be in your source code, but may end up in your state files. Secure state accordingly. – Jake Biesinger Jan 14 '21 at 22:04
  • 2
    @Mar0ux It's working for me. Record the network activity for the oauth flow, then find the GET to "https://slack.com/api/auth.test?token=auth_token" – Jake Biesinger Jan 14 '21 at 22:50
  • I can confirm that this is still working for me as well. To be clear, the request to "https://slack.com/api/auth.test?token=auth_token" happens after you are redirected back to https://console.cloud.google.com – jket Mar 25 '21 at 20:07
  • Still working in June 2022 – AxA Jun 28 '22 at 21:03
  • 1
    Using Chrome network monitoring and searching for hits on: A) search the "auth_token" string (e.g. "auth_token": ****************************************************G7hQ) B) search for the last 4 characters: "G7hQ" C) search for "accessToken" string – AxA Jun 28 '22 at 21:10
  • It still works indeed. I first created the slack notification channel manually using GCP console and at the same time record your network activity. Don't forget to do a test channel. Then I used gcloud to list the channels. That gives you the 4 last characters of the auth_token so that you have an idea what you are looking for. Then you can find one of the last calls the one about testing. In the body of the post it will have the auth_token key in the labels section. – Nick Mar 21 '23 at 20:24
0

One trick is to create your own Slack App and use its oauth-token. Fairly simple if you know how. While researching this problem myself, I found this great blog post that explains it https://bradtho.github.io/technology/gcp-alerting/

Alternatively, you can setup the default Google Cloud Monitoring integration and inspect network calls when clicking on "send test notification". The POST payload contains the token.

pHiL
  • 1,722
  • 18
  • 19