Secret Manager in GCP Explained: Versions, IAM, Rotation, and Best Practices

The simple version

Think of Secret Manager as a locked filing cabinet for your application’s credentials. Only people with the right key can open a specific drawer, and every time someone does, it is logged automatically. When you need to change a credential, you add a new copy to the drawer. Your application finds the newest copy on its own.

Before Secret Manager, teams routinely stored passwords in environment variable files committed to git, passed tokens through Slack messages, or embedded credentials directly in application code. Any of those approaches creates a persistent exposure that is difficult to contain once it happens. Secret Manager gives you one controlled place to store those values, with access gated by IAM and every read logged in audit logs.

What Secret Manager is

Secret Manager is a fully managed GCP service. You do not provision servers, manage databases, or configure encryption. Google handles all of that. You interact with it through the gcloud CLI, client libraries, or the REST and gRPC APIs.

It is designed for values that are:

• Sensitive enough to warrant encryption at rest and access logging
• Referenced by applications rather than by humans in daily work
• Subject to occasional rotation (credential changes)

Common examples include database passwords, third-party API keys, HMAC signing secrets, OAuth client secrets, TLS private keys, and service account key files (though service account keys carry their own risks and should be avoided where possible). What Secret Manager is not designed for is covered further down.

Why Secret Manager matters

Secrets stored in the wrong place are behind a significant proportion of cloud security incidents. The most common failure patterns:

• Secrets in version control. A credential committed to git is compromised from that moment onward, even if deleted later. Git history is permanent, and scanning tools can extract it automatically.

• Hardcoded values in application code. A developer copies a working connection string into source, ships it, and the credential reaches production builds, container images, and eventually a public repository.

• Uncontrolled sharing. Passwords emailed or shared over chat have no audit trail, no expiry, and no access control. You cannot tell who has the credential or revoke access to it.

• Difficult rotation. When credentials are scattered across environment variable files, CI/CD variables, and application configs, rotating them requires touching every location. Often you do not know where all the locations are.

• No separation between environments. Sharing a credential between dev and production means a breach in your development environment exposes production data.

Secret Manager addresses all of these. One controlled location, IAM-gated access, full audit history, and versioning that makes rotation a safe, incremental operation.

How Secret Manager works

The lifecycle from creation to rotation follows a predictable set of steps:

1. Create the secret container. You create a named secret resource and choose a replication policy. No credential value is stored yet.

2. Add a secret version. You add the actual credential value. Secret Manager stores it as version 1, encrypted at rest using Cloud KMS.

3. Grant IAM access. You bind roles/secretmanager.secretAccessor to the service account that your application runs as, scoped to this specific secret.

4. Application reads the secret. The application calls the Secret Manager API using the latest version alias. Secret Manager returns the plaintext value over an encrypted connection to authorised callers only.

5. Rotate by adding a new version. When you need to change the credential, add version 2 with the new value. Applications reading latest automatically pick up the new value on their next read.

6. Disable or destroy the old version. Once you have confirmed the application is using the new value, disable version 1 to make it inaccessible. Destroy it if you want the data permanently deleted. For a detailed walkthrough of this process see Rotating Secrets Automatically.

Version states

Every secret version is in one of three states:

Replication policy

When creating a secret, you choose how Secret Manager distributes it across Google’s infrastructure:

• Automatic replication: Google distributes the secret across multiple regions for high availability, with no configuration required. This is the right choice for most workloads.

• User-managed replication: you specify the exact regions where the secret is stored. Use this when data residency requirements restrict which geographic locations can hold your data. User-managed replication also enables CMEK with a region-specific key via Cloud KMS.

Secret vs secret version

This distinction trips up beginners more than almost anything else in Secret Manager. The two resources have completely different roles:

A practical consequence: you can add 10 versions to a secret and revoke a caller’s access to all of them in one step by removing their binding from the secret resource. You do not need to manage access per version.

IAM roles in Secret Manager

Secret Manager has four main IAM roles. Understanding which one to use and at what scope is the most operationally important part of setting it up correctly. For background on how GCP IAM roles work in general, see the IAM roles guide.

• roles/secretmanager.secretAccessor: allows reading secret version values. Grant this to the service account that your application runs as, scoped to the specific secret it needs. Not at the project level.

• roles/secretmanager.viewer: allows listing secrets and viewing metadata, but cannot read secret values. Useful for operations tooling or dashboards that need to know which secrets exist without accessing their contents.

• roles/secretmanager.secretVersionManager: allows adding new versions and changing version states (enable, disable). Does not allow reading the value of existing versions. Used in automated rotation pipelines where the rotation handler needs to add a new version but should not be able to read existing ones.

• roles/secretmanager.admin: full control, including creating and deleting secrets, managing all versions, and managing IAM bindings. Restrict this to a small number of identities, ideally not attached to running workloads.

When to use Secret Manager

Secret Manager fits whenever an application needs a sensitive value at runtime and you need auditability, rotation, and access control around it. Concrete examples:

• Cloud Run services: inject database passwords, signing keys, or third-party API keys without baking them into container images or Cloud Run environment variables. See the Cloud Run security model for how this integrates with service identity.

• GKE workloads: mount secrets as files using the Secret Manager CSI driver, or sync them into Kubernetes Secrets. Compare with native Kubernetes secret handling in Managing Secrets in Kubernetes.

• Compute Engine VMs: read secrets at startup using the metadata server and the VM’s attached service account, without storing any credentials on the disk image.

• CI/CD pipelines: inject secrets into Cloud Build steps or GitHub Actions without storing them in pipeline YAML. See Secrets in CI/CD Pipelines for patterns and caveats.

• Third-party API credentials: Stripe keys, SendGrid API tokens, Twilio auth tokens, and similar values that need to be available to multiple services but should not be shared through code or config files.

• Per-environment credentials: separate secrets for dev, staging, and production databases, with access granted to only the service accounts in the corresponding environment.

When not to use Secret Manager

Secret Manager is not a general-purpose tool for all configuration:

• Non-sensitive application config. Feature flags, log levels, timeout values, and similar settings do not need encryption or access auditing. Environment variables, parameter files, or a config service are more appropriate.

• Encrypting large files or data. Secret Manager is for small credential values (under 64 KB per version). For encrypting arbitrary data at rest, use Cloud KMS directly.

• Managing cryptographic keys. Secret Manager stores complete values and returns them in plaintext to authorised callers. Cloud KMS stores key material and performs cryptographic operations without exposing the raw key. If you need to encrypt or sign data without the key ever leaving Google’s infrastructure, that is a Cloud KMS use case, not Secret Manager.

Secret Manager vs alternatives

Secret Manager vs environment variables

Environment variables are quick to set up but carry real risks for sensitive values. They are not encrypted at rest, not versioned, not audited, and can appear in crash dumps, debug output, and process listings. Secret Manager encrypts values at rest, logs every access, and makes rotation possible without redeployment. For anything sensitive, Secret Manager is the better choice. For non-sensitive config, environment variables are fine.

Secret Manager vs Cloud KMS

These solve different problems. Secret Manager stores a complete value and returns it in plaintext to authorised callers. Cloud KMS stores cryptographic key material and performs encrypt/decrypt/sign operations without ever returning the raw key. Secret Manager actually uses Cloud KMS under the hood to encrypt secret versions at rest. Choosing between them is not an either/or decision. Store credentials in Secret Manager and use Cloud KMS when you need to encrypt your own data with customer-managed keys.

Secret Manager vs Kubernetes Secrets

Native Kubernetes Secrets are base64-encoded, not encrypted by default (unless you configure etcd encryption), and are scoped to a single cluster. They have no versioning and no centralised audit trail. Secret Manager integrates with GKE through the Secret Manager CSI driver, which mounts secret values as files in your pods without requiring you to manage Kubernetes Secrets at all. For GKE workloads the recommended pattern is to use Secret Manager via Workload Identity rather than native Kubernetes Secrets. For the detailed comparison see Managing Secrets in Kubernetes.

Creating and accessing secrets

Create a secret and add the first version:

# Create the secret container with automatic replication
gcloud secrets create db-password \
  --replication-policy=automatic \
  --project=my-app-prod

# Add the first version (the actual credential value)
echo -n "my-super-secret-password" | gcloud secrets versions add db-password \
  --data-file=- \
  --project=my-app-prod

Grant a service account access to read this specific secret:

gcloud secrets add-iam-policy-binding db-password \
  --member="serviceAccount:api-server@my-app-prod.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor" \
  --project=my-app-prod

Read the secret value:

# Read the latest version
gcloud secrets versions access latest \
  --secret=db-password \
  --project=my-app-prod

# Read a specific version by number
gcloud secrets versions access 2 \
  --secret=db-password \
  --project=my-app-prod

Manage version states during and after rotation:

# Disable an old version (data preserved, temporarily inaccessible)
gcloud secrets versions disable 1 \
  --secret=db-password \
  --project=my-app-prod

# Permanently destroy a version
gcloud secrets versions destroy 1 \
  --secret=db-password \
  --project=my-app-prod

Accessing secrets from application code

Applications running on GCP access secrets using the Secret Manager client library and the service account identity attached to the workload. Your application never needs to store a credential to retrieve a credential. The workload identity is the authentication mechanism. There are two main access patterns:

Read at startup

Read the secret once when the application starts and store the value in memory. This is simple and adds no per-request latency, but requires a restart to pick up rotated values.

from google.cloud import secretmanager

client = secretmanager.SecretManagerServiceClient()
name = "projects/my-app-prod/secrets/db-password/versions/latest"
response = client.access_secret_version(request={"name": name})
password = response.payload.data.decode("UTF-8")

Mount as a volume (Cloud Run and GKE)

Secret Manager integrates with Cloud Run and GKE to mount secret values as files in the container filesystem. The platform handles the API call; your application just reads a file. When a new secret version is added, the mounted file updates automatically, allowing zero-downtime rotation if the application watches the file for changes.

Rotation and automated rotation

Manual rotation

The simplest rotation process: generate a new credential value, add it as a new version, verify your application is reading the new value, then disable or destroy the old version. If your application reads latest, no code change is needed. Just add the new version.

Automated rotation via Pub/Sub

For credentials that need scheduled rotation, Secret Manager can publish a notification to a Pub/Sub topic when a rotation window arrives. A Cloud Function or Cloud Run service subscribed to that topic handles the rotation automatically:

1. Secret Manager publishes the rotation notification to your Pub/Sub topic
2. Your rotation handler is triggered
3. The handler generates a new credential (e.g. rotates a database password)
4. The handler adds the new value as a new secret version
5. The handler updates the upstream system (e.g. updates the database user password)
6. The handler disables the old version

# Configure rotation notifications on an existing secret
gcloud secrets update db-password \
  --rotation-period=7776000s \
  --next-rotation-time=2026-06-01T00:00:00Z \
  --topics=projects/my-app-prod/topics/secret-rotation \
  --project=my-app-prod

Because the rotation handler adds a new version and your application reads latest, the cutover happens automatically on the next secret read, with no redeployment. For the complete implementation including the Cloud Function handler and Terraform setup, see Rotating Secrets Automatically.

Best practices

• Grant secretAccessor at the secret level, not the project level. Scope access as tightly as the workload requires.

• Create one secret per service per environment. Separate dev, staging, and prod credentials. Separate credentials per service.

• Use latest with a tested rotation process. The alias makes rotation seamless, but validate that your rotation handler works before relying on it in production.

• Audit access regularly. Secret Manager writes to Data Access audit logs. Review access patterns in Cloud Audit Logs to detect unexpected or anomalous reads.

• Document secret ownership. Use labels or a runbook to track which team owns each secret, what it is used for, and what the rotation policy is.

• Rotate high-risk credentials on a schedule. Database passwords and signing secrets should rotate on a fixed cadence, not just when something goes wrong.

• Prefer Workload Identity over service account keys. If your workload runs on GCP infrastructure, it can authenticate to Secret Manager using its attached service account identity without needing a stored key file. See Workload Identity Federation for workloads outside GCP.