0

I've pipeline in project A (pA) which reads from DB2 and try to write in BigQuery of project B (pB), when I try with the validate option into BigQuery sink (before deploy) got this message error:

Failed to configure pipeline: Stage 'BigQuery' encountered:
Unable to get details about the BigQuery table:
Access Denied: Table pA:dataset_1.table_X:
Permission bigquery.tables.get denied on table pA:dataset_1.table_X (or it may not exist).

Table schema was created on pB (target), and secondly both SA of Data Fusion instance and SA with nomenclature [number-prject-compute@developer.gserviceaccount.com] in pA contains follows roles:

What other role do I need in the SA of pA?

Service Accounts of the project A and Roles over Project B

Message error on BigQuery properties

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Lais T
  • 17
  • 4
  • Please provide enough code so others can better understand or reproduce the problem. – Community Oct 07 '21 at 12:37
  • From the error message it seems like service account being used does not have access to get bq tables for project P2. – vinisha Oct 11 '21 at 22:18
  • @vinisha the SA of instance data fusion of project A has this roles over project B: • Cloud Data Fusion API Service Agent • BigQuery Data Editor • BigQuery Data Viewer • BigQuery Job User • BigQuery Metadata Viewer • Storage Object Creator • Storage Object Viewer The issue here is, what SA I should configure with this roles?, the services account of data fusion instance or SA of compute engine or another? – Lais T Oct 15 '21 at 23:47
  • So the SA with which Dataproc cluster/job is being created should have permissions. – vinisha Oct 18 '21 at 17:56
  • Still I've this error, I have not yet deployed, the issue is on the Validate option. I attach an image of the roles of the SA of project A has over BigQuery on project B, I'm using the SA's indicated in the post "https://stackoverflow.com/questions/61726314/connect-bigquery-as-a-source-to-data-fusion-in-another-gcp-project ", is there any other service account that I should consider? I added new roles to SA1 and SA2, I'm validating the BigQuery Properties and get the above error. – Lais T Oct 26 '21 at 22:25
  • I solved this issue, data fusion are using two service accounts, the required roles are setting to the instance data fusion SA: cloud-datafusion-management-sa@i798133b6XXXXXXX-tp.iam.gserviceaccount.com and to the second SA for data fusion: service-[project-numbre]@gcp-sa-datafusion.iam.gserviceaccount.com, I don't know why exist two SA's for the same data fusion instance, but work fine. – Lais T Dec 02 '21 at 00:02

0 Answers0