GCP GKE - Google Compute Engine: Not all instances running in IGM

Question

Turned out this is Terraform issue Terraform google_project_iam_binding deletes GCP compute engine default service account from IAM principals

---

Original Question below

Trying to create a GKE cluster as the user having the Owner role. However it fails with the error message. Deleted and retried a few times but the same error.

Please advise how to troubleshoot and understand the cause.

Users

Login user

Service Accounts

GKE Service Account setting

Error

Google Compute Engine: Not all instances running in IGM after 15.945831085s. 
Expected 3, running 0, transitioning 3. 
Current errors: 
[PERMISSIONS_ERROR]: Instance 'gke-cluster-1-default-pool-a3bd7bfb-cw7n' creation failed: Required 'compute.instances.create' permission for 'projects/412177242019/zones/us-central1-c/instances/gke-cluster-1-default-pool-a3bd7bfb-cw7n' (when acting as '412177242019@cloudservices.gserviceaccount.com'); 
[PERMISSIONS_ERROR]: Instance 'gke-cluster-1-default-pool-a3bd7bfb-cw7n' creation failed: Required 'compute.disks.create' permission for 'projects/412177242019/zones/us-central1-c/disks/gke-cluster-1-default-pool-a3bd7bfb-cw7n' (when acting as '412177242019@cloudservices.gserviceaccount.com'); 
[PERMISSIONS_ERROR]: Instance 'gke-cluster-1-default-pool-a3bd7bfb-cw7n' creation failed: Required 'compute.disks.setLabels' permission for 'projects/412177242019/zones/us-central1-c/disks/gke-cluster-1-default-pool-a3bd7bfb-cw7n' (when acting as '412177242019@cloudservices.gserviceaccount.com'); 
[PERMISSIONS_ERROR]: Instance 'gke-cluster-1-default-pool-a3bd7bfb-cw7n' creation failed: Required 'compute.subnetworks.use' permission for 'projects/412177242019/regions/us-central1/subnetworks/default' (when acting as '412177242019@cloudservices.gserviceaccount.com'); 
[PERMISSIONS_ERROR]: Instance 'gke-cluster-1-default-pool-a3bd7bfb-cw7n' creation failed: Required 'compute.subnetworks.useExternalIp' permission for 'projects/412177242019/regions/us-central1/subnetworks/default' (when acting as '412177242019@cloudservices.gserviceaccount.com') (truncated).

Update

Added the "roles/compute.admin" to the service account.

$ gcloud iam service-accounts list
DISPLAY NAME                            EMAIL                                                                    DISABLED
Compute Engine default service account  412177242019-compute@developer.gserviceaccount.com                       False

$ gcloud projects add-iam-policy-binding 'positive-theme-323611' --member=serviceAccount:412177242019-compute@developer.gserviceaccount.com --role='roles/compute.admin'
Updated IAM policy for project [positive-theme-323611].
bindings:
...
- members:
  - serviceAccount:412177242019-compute@developer.gserviceaccount.com
  role: roles/compute.admin
...

However, still having the same issue.

Google Compute Engine: Not all instances running in IGM after 18.269931718s. Expected 3, running 0, transitioning 3. Current errors: [PERMISSIONS_ERROR]: Instance 'gke-cluster-4-default-pool-289807f2-5dh5' creation failed: Required 'compute.instances.create' permission for 'projects/412177242019/zones/us-central1-c/instances/gke-cluster-4-default-pool-289807f2-5dh5' (when acting as '412177242019@cloudservices.gserviceaccount.com'); [PERMISSIONS_ERROR]: Instance 'gke-cluster-4-default-pool-289807f2-5dh5' creation failed: Required 'compute.disks.create' permission for 'projects/412177242019/zones/us-central1-c/disks/gke-cluster-4-default-pool-289807f2-5dh5' (when acting as '412177242019@cloudservices.gserviceaccount.com'); [PERMISSIONS_ERROR]: Instance 'gke-cluster-4-default-pool-289807f2-5dh5' creation failed: Required 'compute.disks.setLabels' permission for 'projects/412177242019/zones/us-central1-c/disks/gke-cluster-4-default-pool-289807f2-5dh5' (when acting as '412177242019@cloudservices.gserviceaccount.com'); [PERMISSIONS_ERROR]: Instance 'gke-cluster-4-default-pool-289807f2-5dh5' creation failed: Required 'compute.subnetworks.use' permission for 'projects/412177242019/regions/us-central1/subnetworks/default' (when acting as '412177242019@cloudservices.gserviceaccount.com'); [PERMISSIONS_ERROR]: Instance 'gke-cluster-4-default-pool-289807f2-5dh5' creation failed: Required 'compute.subnetworks.useExternalIp' permission for 'projects/412177242019/regions/us-central1/subnetworks/default' (when acting as '412177242019@cloudservices.gserviceaccount.com') (truncated).

Related

• Building a GKE cluster on GCP with Terraform permission error

Answer 1

You are facing the permission error because the service account does not have the correct IAM permission. As per given information you have added the compute.admin role to the compute engine default service account but not to this service account 412177242019@cloudservices.gserviceaccount.com.

Service Account User grants a Google Cloud user account the permission to perform actions as though a service account were performing them.

• Granting the iam.serviceAccountUser role to a user for a project gives the user all the roles granted to all service accounts in the project, including service accounts that may be created in the future.

• Granting the iam.serviceAccountUser role to a user for a specific service account gives a user all the roles granted to that service account.

Service accounts are identities, you can let a service account access resources in your project by granting it a role, just like you would for any other principal. This service account should have the Editor role, which provides a broad set of permissions.The service account you are using doesn’t have the required ServiceAccountUser role (roles/iam.serviceAccountUser) and Editor role (roles/editor).