Tracking the status of GCP Service Accounts to avoid failure in terraform deployments

As the owner of the data platform on GCP, my team provides access to various Service Accounts on different services (e.g., BigQuery datasets, GCS buckets, etc.). Access provisioning happens through Terraform and GitHub Actions workflow.

These GCP Service Accounts may be created in other GCP projects, not under our control or scope. The problem is that we will know that a Service Account is deleted only when our terraform deployment fails. This is frustrating for us because of the scale we operate at. To handle this situation, I developed a process to track the status of the Service Accounts and send an email notification to my team.

In my previous blog post, I explained the process I developed to capture the IAM permissions granted across multiple GCP projects and store the details in a BigQuery table. This BigQuery table came in handy for the new process I developed for tracking the status of the Service Accounts. I get the list of Service Accounts (to which we have granted access) from the BigQuery table, and then run “gcloud iam service-accounts describe” command on that list.

Note:

1. There are 3 different types of GCP Service Accounts. In the SQL statement I coded, I have excluded all the “service default” and “service agent” service accounts. The script is running only on “user-managed “ service accounts.
2. The Service Account that runs the script needs these permissions at GCP org level (as the Service Accounts that have access may be created in other projects).

iam.serviceAccounts.get
iam.serviceAccounts.list
iam.serviceAccounts.getIamPolicy
resourcemanager.projects.get
resourcemanager.projects.list
resourcemanager.projects.getIamPolicy

Disclaimer: The posts here represent my personal views and not those of my employer or any specific vendor. Any technical advice or instructions are based on my own personal knowledge and experience.