I would like to see if any abnormal data loading issues in Bigquery tables through alerts based on some rules, Any possibility to do that with cloud logging and alert policies ?
Asked
Active
Viewed 246 times
0
-
What do you mean by abnormal? Is it number of rows, range of values, or what else do you have in your mind? – Sabri Karagönen Feb 17 '23 at 07:00
-
number of inserts /updates more than mean value over 3-6 months etc. – Harshith S H Feb 17 '23 at 07:06
1 Answers
1
Your question is a bit unclear, are you looking for notifications about the table itself (load times / size etc) or what is in the table (the data)?
For what the table itself has already answered it here BigQuery - Scheduled Query Update Notification Email
If its what is in the table you are better off using python to automate this, there are plenty of tutorials about how to link BigQuery to Python and automate it and below is a similar query that should point you on the right track for automating an email from a condition How to send email alert through python if a string is found in a csv file?.
Max_Stone
- 254
- 5
-
Agree with Max, I would also suggest looking into cloud monitoring. There are a lot of BQ metrics around which you can build alerts. – Luka Feb 17 '23 at 16:22