So, I wrote a query which then I saved into a View by clicking 'Save View' button in Bigquery. With the same query, I schedule them into a new table by clicking 'Schedule queries' ( I am aware that this is a Beta version ). Couple days later I compare the data (View vs Table) and it shows that the data in Table has duplication.
Asked
Active
Viewed 936 times
0
-
Can you share more details with us? Can you show us the query you ran? Do you think the query ran more than once? – Felipe Hoffa Jan 23 '19 at 01:32
1 Answers
2
You can set the write preference on a scheduled query so that it will overwrite the destination table, or append to it, depending on your requirements.
If you require something more finessed, you can use BigQuery Data Definition Language (DDL) so that your query creates new tables or partitions etc. again depending on your specific requirements.
If you are seeing duplicate data, and you can't work out why, you could also schedule a second query to look through your table and remove duplicate entries.
Ben P
- 3,267
- 4
- 26
- 53