5

Google recently released the new Eventarc API trigger for e.g Cloud run. I had the idea to build one trigger for my cloud storage like: new file in bucket → trigger cloud run (with audit log trigger)

cloud_run_path: ...run.app/api/v1/data-fetcher bucket_id: test-bucket

I just created the trigger with the following command and it is successful:

gcloud beta eventarc triggers create test-event-trigger \
--location=europe-west1 \
--destination-run-service=test-event-data-fetcher \
--destination-run-path=/api/v1/data-fetcher \
--destination-run-region=europe-west1 \
--matching-criteria="type=google.cloud.audit.log.v1.written" \
--matching-criteria="serviceName=storage.googleapis.com" \
--matching-criteria="methodName=storage.objects.create" \
--matching-criteria="resourceName=projects/_/buckets/test-bucket" \
--service-account=$PROJECT_NR-compute@developer.gserviceaccount.com

The problem is, I don't want the trigger to look for new files in all buckets in the project, just for one specific bucket (e.g test-bucket). I tested now several options with different writings (with :, =~, ...), but the trigger don't accept these. Maybe you can help me out with the syntax or show me way how its possible to create a Trigger for one specific bucket in my project? Like this it's not working...

Travis Webb
  • 14,688
  • 7
  • 55
  • 109
Rookez
  • 51
  • 5
  • Can you check if you haven't other triggers configured? – guillaume blaquiere Nov 04 '20 at 16:45
  • You mean, that they maybe block each other somehow? I just have a cloudbuild trigger running in this project – Rookez Nov 04 '20 at 17:44
  • No a Cloud Build trigger, an event trigger. What's the response of this command `gcloud beta eventarc triggers list --location=europe-west1`? – guillaume blaquiere Nov 04 '20 at 19:46
  • Its just the one i created above and the state is on active. – Rookez Nov 05 '20 at 07:03
  • Do you receive something with your configuration? – guillaume blaquiere Nov 06 '20 at 09:35
  • What error do you get? – Kolban Nov 08 '20 at 20:03
  • Looking at an actual Audit record that is generated when a new object is created in a bucket, we see that the ce-resourcename Cloud Events header that is present includes the object that was created. For example: projects/_/buckets/[BUCKET]/objects/[OBJECT] It isn't clear to me that the --matching-criteria supports any form of wildcarding or prefix matching. If it doesn't support that, then your Cloud Run will be invoked for every object created in all your buckets and it would be necessary for your Cloud Run logic to only process desired events. – Kolban Nov 08 '20 at 20:11
  • Okay yes that was my problem, so there is no way to set a wildcard i think... Hopefully they will add this soon or in the run of this beta – Rookez Nov 09 '20 at 21:13

2 Answers2

4

As of this time (2020-11) wild cards and prefix matching are not supported in the configuration of the trigger. It appears that this feature has been heavily requested and is apparently known to the product manager at Google that owns this product area. There is no public/committed date for when such a feature will be added. If this is a blocker for you, contact your local Google rep and they can schedule a call with the Product Manager to discuss the road map.

Kolban
  • 13,794
  • 3
  • 38
  • 60
0

You can achieve this by using the matching criteria type=google.cloud.pubsub.topic.v1.messagePublished which subscribes to a particular Pubsub topic instead of all storage events from the audit logs. Then, configure a storage trigger on test-bucket to publish to the topic generated for this Eventarc trigger.

Edit: my configuration is below.

gcloud beta eventarc triggers create <my_trigger> \
    --destination-run-service=<my service> \
    --destination-run-region=us-central1 \
    --destination-run-path="/<my_endpoint>" \
    --matching-criteria="type=google.cloud.pubsub.topic.v1.messagePublished" \
    --service-account=${PROJECT_NUMBER}-compute@developer.gserviceaccount.com

then run

gcloud beta eventarc triggers describe <my_trigger>

to get the name of the generated topic, and conigure your storage triggers to publish to that topic.

Travis Webb
  • 14,688
  • 7
  • 55
  • 109