0

Has anybody had any success in applying a run_date parameter when creating a Transfer in BigQuery using the Transfer service UI ?

I'm taking a CSV file from Google Cloud storage and I want to mirror this into my ingestion date partitioned table, table_a.

Initally I set the destination table as table_a, which resulted in the following message in the job log:

Partition suffix has to be set for date-partitioned tables. Please recreate your transfer config with a valid table name. For example, to load new files to partition of the run date, specify your table name as transferTest${run_date} for daily partitioning or transferTest${run_time|"%Y%m%d%H"} for hourly partitioning.

I then set the destination to table_a$(run_date), which then issues the warning:

Invalid table name. Please use only alphabetic, numeric characters or underscore with supported parameters wrapped in brackets.

However it won't accept table_a_(run_date) either - could anyone please advise?

best wishes

Dave

Dave Papworth
  • 313
  • 1
  • 3
  • 10
  • 1
    I ran into something possibly related: https://stackoverflow.com/questions/65797659/bigquery-scheduled-data-transfer-throws-incompatible-table-partitioning-specifi/65852779#65852779 – mraguda Jan 22 '21 at 21:55

1 Answers1

0

Apologies - i've identified the correct syntax now

table_a_{run_date}

Dave Papworth
  • 313
  • 1
  • 3
  • 10