Stack Overflow
Stack Overflow

Questions tagged [google-dataplex]

google-dataplex relates to questions about Google Cloud Dataplex (https://cloud.google.com/dataplex).

17 questions
1
vote
2 answers

Insufficient CPU quota when running data quality task in GCP dataplex

I try to follow the below guide from GCP to create a data quality task. https://cloud.google.com/dataplex/docs/check-data-quality?&_ga=2.139058355.-143783534.1669209779#before_you_begin when I try to run the task I get the error As per my…
user546298
  • 41
  • 4
1
vote
3 answers

Can a data source in GCP(cloud storage/bigquery table) be assigned as an asset to multiple data lakes or zones, in dataplex?

I have created a big query table, and attached it as an asset, in one of my data lake's zones. I want to attach the same data source as an asset in a different zone, or even a different lake. But, keep running into the error 'Provided resource…
Dhruv
  • 11
  • 1
0
votes
0 answers

Query job is not working in Dataplex Exploration Workbench

I'm not able querying the data over exploration workbench of dataplex, even i have added assets in dataplex zone and that entities are visible in explore tab but after running the simple select * from production1raw.machine_data; query is not…
Pooja Khaire
  • 1
  • 1
0
votes
0 answers

Constraint constraints/compute.requireOsLogin violated for project (project id)

While creating the Data Quality task in dataplex i am facing the issue as Constraint constraints/compute.requireOsLogin violated for project. I have check all the task configuration but i am not able to find anything related this error.
Sheikh Mubashir
  • 1
0
votes
1 answer

Grant access on BigQuery based on table prefix

I need to grant a group of users conditional access as bigquery.dataEditor on tables that start with a certain prefix. Example : The group should only have access to tables starting with the word "alpha_" Is it possible to propagate this condition…
Snowfire
  • 171
  • 9
0
votes
1 answer

Dataplex Data Quality Rules

I am looking for a google native option for data quality and went through Dataplex in GCP world. However there are 2 ways to define data quality rules in Dataplex - i) via process & ii) via Govern. What is the difference between Dataplex Data…
Gora Bhattacharya
  • 1
  • 1
  • 4
0
votes
1 answer

How can I retrieve dataplex data profile results to BigQuery?

I have been exploring google cloud dataplex data profiling feature here I followed the steps given in creating the data profile scans and noticed the profile results are shown in the Profile section as per this documentation If I want to retrieve…
PK_gcp
  • 29
  • 5
0
votes
0 answers

Table DataPlex not supported by Bigquery

Currently I created a data lake in dataplex with its zone, within it I generated the asset and the tables are reflected, now when I look for it and access a table it tells me that it is not compatible with bigquery and it does not show me the link…
Nahuel Alvarez
  • 1
  • 1
0
votes
0 answers

Error: Data masking cannot be applied to table "table-name" on field "email" as the field is used for partitioning or clustering

Steps to reproduce this error: Step 1: I have a Big query which has couple of columns & "email" column is clustered. Step 2: I have also applied Policy tagging (dynamic data masking) on this clustered column. Step 3: Post applying I am unable to…
Viknesh S K
  • 101
  • 3
  • 9
0
votes
0 answers

GCP data catalog/dataflex - can I register Kafka topics and on-prem databases?

Is it possible to also register/add metadata of Kafka topics/producers/consumers and on-premise databases etc in GCP data catalog/Dataplex? Could someone please point me to some examples how to do the same? Thanks in advance
Ashok
  • 5
  • 2
0
votes
1 answer

Can BigQuery views be listed as entities under BQ Dataset assets in GCP Dataplex?

In the current usecase, I am using authorized views to handle row level security in the BigQuery dataset and Dataplex to handle the governance and permissions. When a BQ Dataset is added as an asset, only the tables under it are getting listed as…
Padmasaran S
  • 1
  • 2
0
votes
2 answers

Dataplex API Tag Policies

I'm on exploration Dataplex API with Python in Google Documentation, there is documentation to Get Lake, Zone, Assets, etc. I've explored that documentation, but I didn't find any documentation related to Tag Policies, for example, I need to attach…
MADFROST
  • 1,043
  • 2
  • 11
  • 29
0
votes
1 answer

Dataplex Explore Workbench how is works?

i´m trying to run some data exploration via Dataplex Explore workbench, however i can´t pass this first page: i have created a Lake, Zone and Assets: also a Dataproc Metastore cluster: and 'default' env for the lake: but the explore tab is still…
Cir02
  • 107
  • 7
0
votes
1 answer

How to update the Data Catalog tag values through airflow CloudDataCatalog operators

I would like to update Data Catalog tags through airflow CloudDataCatalog operators and schedule the DAG to update the tag values daily. This is to capture the ETL status on a BigQuery table and feed that status to Data Catalog tag. Could you…
Muthu lakshmi
  • 1
0
votes
0 answers

Is Data Scan in Dataplex available for Americas São Paulo?

I tried to create a profile in Data Scan, a PRE-OFFERING in Dataplex, but even having the Admin permisssions, an error occured making it impossible to test data scan. I submitted a feedback but I still have no answer about it. Can anyone help? I…
Henrique Anjos
  • 1
1
2