0

We are fetching data from Appsflyer source to a Google cloud storage bucket

Then through data transfer service - trying to insert in gbq table

Data transfer service is running successfully but data is not shown in GBQ table, but whenever we are creating a new data transfer service all historical data is getting loaded but not after that point. So everytime we have to manually drop that table and create new data transfer service to load all historical data.

I can see in logs that data transfer service is running successfully. I have checked data is getting updated in bucket as well.

Expecting data to update in gbq table on every successul run.

  • FYI - I had asked ChatGPT and got following reasons & validated all of these Check the data transfer schedule: Make sure that the data transfer schedule is set up correctly. If the data transfer is set to run only once or at specific intervals, it may not be picking up new data after the initial historical load. Make sure that the schedule is set to run continuously or at a frequency that matches your data ingestion requirements. – mh khan Mar 28 '23 at 12:56

0 Answers0