Automatic Secret Rotation in GCP: Secret Manager, Pub/Sub, and Safe Rollovers
Automatic secret rotation means replacing a credential on a schedule and safely rolling it out to every consumer before retiring the old value. In Google Cloud, Secret Manager can trigger rotation notifications via Pub/Sub, but it does not generate new credentials by itself. You still write the rotation logic. The reward for getting this right: a shorter exposure window when credentials are compromised, and rotation that runs consistently without anyone having to remember to do it.
Simple explanation
Think of a secret like a padlock combination. Today it is 1234. Next month it should be 5678. You cannot just change the combination and walk away. You need to give everyone with legitimate access the new combination first, confirm they can use it, and only then invalidate the old one.
Automatic rotation in GCP is a system that does this on a schedule. Secret Manager fires a signal when rotation time arrives. Your code receives that signal, generates a new credential, hands it to the right services, confirms everything still works, and retires the old value. You write the code once; the schedule runs it automatically from then on.
Rotating a secret is like re-keying a lock. You cut new keys first, distribute them to everyone who needs access, confirm everyone can get in, and only then retire the old key and change the lock cylinder. Doing it the other way around locks everyone out during the changeover.
Why automatic secret rotation matters
A secret that is never rotated is permanently compromised from the moment it leaks, whether that leak is detected or not. Rotation does not prevent leaks, but it bounds the damage. A database password rotated every 30 days limits the worst-case window of unauthorised access to 30 days, even if the credential was silently exfiltrated weeks ago.
Reduced exposure window. Short-lived credentials limit how long a compromised secret remains valid. An attacker who steals a credential that rotates in 48 hours has a much narrower attack window than one that never changes.
Operational consistency. Manual rotation is a multi-step process spread across multiple systems. Steps get missed. Automation makes every rotation identical, removing the human error surface from one of the most sensitive operations in your infrastructure.
Compliance and audit readiness. PCI-DSS, SOC 2, and ISO 27001 require periodic credential refresh for certain credential types. Automated rotation with Secret Manager gives you a verifiable audit trail in Cloud Audit Logs that satisfies auditor requirements without manual record-keeping.
Smaller blast radius. If a long-lived shared credential leaks, the damage can be enormous. Automated per-service rotation with short-lived credentials shrinks that blast radius significantly.
How automatic secret rotation works in GCP
Secret Manager does not rotate credentials by itself. What it does is send a notification at the right time so your automation can act. Here is the full flow from trigger to retirement:
Rotation window arrives. The rotation period you configured on the secret elapses, for example every 90 days.
Secret Manager publishes a notification. A message is sent to the Pub/Sub topic attached to the secret. The message contains the secret name, project, and event type.
Rotation handler receives the event. A Cloud Function or Cloud Run service subscribed to that topic is triggered automatically.
New credential is generated. Your rotation handler calls the external system to generate a new credential at the source: the database, the third-party API, the OAuth provider.
New version is stored in Secret Manager. The handler adds the new credential as a new secret version. The old version stays
ENABLED.Consumers are updated. Depending on how each application reads secrets, consumers may pick up the new
latestversion automatically, or they may need a restart or redeployment.Health checks pass. The handler confirms that consuming applications are functioning correctly with the new credential before anything is retired.
Old credential is retired. Once health checks pass, the old version is disabled in Secret Manager and the old credential is revoked at the source system.
What GCP handles vs what you write
A common source of confusion is which parts of this flow are built-in GCP features and which are your responsibility.
- The rotation schedule on the secret resource
- Publishing the Pub/Sub notification when the schedule fires
- Delivering the message to your subscriber
- Audit logging of every secret access and version change
- Receive the Pub/Sub event and parse it
- Call the downstream system to generate a new credential
- Add the new credential as a new secret version
- Update consumers and run health checks
- Disable the old version once all consumers have migrated
Secret Manager will not generate a new database password for you. It will tell you, via Pub/Sub, that it is time to do so. This design is flexible: the same notification mechanism works for database passwords, API keys, TLS certificates, and any other credential type, because you control the handler logic.
The rotation handler needs a
service account
with roles/secretmanager.secretVersionManager to add new
versions and roles/secretmanager.secretVersionDestroyer to
disable old ones. Scope these roles to the specific secret, not at project
level. See the
principle of least privilege
guide for how to do this correctly.
Automatic rotation vs manual rotation
| Automatic | Manual | |
|---|---|---|
| Trigger | Schedule-driven, fires without human action | Human remembers — or forgets |
| Speed | Runs within seconds of the rotation window | Depends on who is on-call and when |
| Error risk | Consistent once the handler is well-tested | High — multi-step process, easy to miss a consumer |
| Consistency | Identical every time | Varies by operator and fatigue level |
| Operational effort | High upfront, near-zero ongoing | Low upfront, high ongoing |
| Audit trail | Automatic via Cloud Audit Logs | Relies on manual documentation |
| Best for | Production systems, regulated environments, frequent rotation | One-off or infrequent rotations in low-risk environments |
When to use automated rotation
Automated rotation is worth the setup cost when the credential is sensitive, used frequently, or subject to a compliance rotation requirement.
Database passwords for Cloud SQL, AlloyDB, or any external database used by Cloud Run or GKE workloads.
Third-party API keys used by backend services: payment providers, email platforms, analytics APIs, and similar.
Service-to-service credentials in microservice architectures where a static shared secret is the only authentication option available.
CI/CD pipeline tokens. See
Secrets in CI/CD Pipelines
for the broader pattern. Pipeline tokens that are never rotated tend to accumulate stale access over time.
Regulated environments where PCI-DSS, SOC 2, HIPAA, or similar frameworks mandate periodic credential refresh.
If a credential is low-sensitivity, rotated infrequently, and the consumer can be updated in minutes, a documented manual runbook may be simpler and safer than building and maintaining a rotation handler for it. Save the automation effort for secrets that matter most.
Step-by-step rotation process
Rotation is a two-phase workflow. The phases must be kept separate: retiring the old credential before confirming all consumers are on the new one will cause an outage.
Phase 1: Add the new version
Generate a new credential at the source system: a new database password, a new API key, a new token from the provider.
Add the new value as a new secret version in Secret Manager. The previous version stays
ENABLED.Both the old and new credentials are now valid simultaneously. This is intentional. It is the overlap window.
Phase 2: Propagate and retire
Update consuming applications to use the new version. Applications referencing
latestmay pick this up automatically on the next read; others need a restart or redeployment.Run health checks. Call an application endpoint that exercises the credential path to confirm the new value works end-to-end.
- Once health checks pass, revoke the old credential at the source system.
- Disable the old secret version in Secret Manager.
After a further rollback window (typically a few days), destroy the old version to remove it permanently.
The overlap window must be long enough for all consumers to migrate. For applications that restart slowly, update only on the next deployment, or hold long-lived authenticated connections, this can be hours or days. Watch your Cloud Audit Logs during this period to see which version each caller is actually reading. Do not proceed to Phase 2 until every active caller has moved.
Setting up rotation with Secret Manager and Pub/Sub
The following commands create a Pub/Sub topic to receive rotation notifications, grant Secret Manager permission to publish to it, and configure a 90-day rotation schedule on an existing secret.
# Create a Pub/Sub topic for rotation events
gcloud pubsub topics create secret-rotation \
--project=my-app-prod
# Grant Secret Manager's service agent permission to publish to the topic
# Replace PROJECT_NUMBER with your GCP project number (not the project ID)
gcloud pubsub topics add-iam-policy-binding secret-rotation \
--member="serviceAccount:service-PROJECT_NUMBER@gcp-sa-secretmanager.iam.gserviceaccount.com" \
--role="roles/pubsub.publisher" \
--project=my-app-prod
# Configure a 90-day rotation schedule on an existing secret
# 7776000s = 90 days in seconds
gcloud secrets update db-password \
--rotation-period=7776000s \
--next-rotation-time=2026-06-01T00:00:00Z \
--topics=projects/my-app-prod/topics/secret-rotation \
--project=my-app-prodOnce configured, Secret Manager publishes a message to the topic every 90 days. Your Cloud Function or Cloud Run service subscribed to that topic handles the rest: generating the new credential, storing it as a new version, and managing the rollout.
Safe versioning and rollout
Whether rotation is automated or manual, the safe versioning steps are the same: add the new version first, verify it, update consumers, then and only then disable the old one.
# Add a new version — old version stays ENABLED
echo -n "new-password-value" | gcloud secrets versions add db-password \
--data-file=- \
--project=my-app-prod
# List versions to confirm the new version is now the latest
gcloud secrets versions list db-password \
--project=my-app-prod
# Read back the latest version to confirm the value looks correct
gcloud secrets versions access latest \
--secret=db-password \
--project=my-app-prod
# After consumers are updated and health checks pass:
# Disable the old version (version 1 in this example)
gcloud secrets versions disable 1 \
--secret=db-password \
--project=my-app-prod
# After a rollback window has passed, destroy the old version permanently
gcloud secrets versions destroy 1 \
--secret=db-password \
--project=my-app-prodApplications that cache the secret value at startup continue using the old
credential until they restart. Cloud Run services that mount secrets as
environment variables need redeployment; those that mount secrets as volume
files receive the new value automatically once the new version is
latest. Know each consumer’s behaviour before you
disable the old version. See
Managing Secrets in Kubernetes
for how GKE workloads handle secret version updates.
Observability and validation during rotation
A rotation that appears complete but left some consumers on the old credential will cause intermittent failures after the old version is disabled. Build validation into your rotation workflow before you retire anything.
Health check endpoints. After updating consumers, call an application endpoint that exercises the credential path to confirm the new value works end-to-end before retiring the old one.
Cloud Audit Logs, Data Access. Query Cloud Audit Logs for your secret to see which version each caller is reading. If anything is still requesting version 1 after you believed all consumers migrated, it will appear here.
Application error rates. Watch authentication failure metrics and database connection error rates during the overlap window. A spike in auth failures is a clear signal that a consumer has not yet adopted the new version.
Rotation handler logs. Ensure your rotation handler emits structured log events for each step: not the secret value itself, but status events like “new version added”, “health check passed”, “old version disabled”. These logs make debugging a failed rotation much faster.
Rotation scripts commonly contain debugging log statements. Ensure none of them output the actual secret value. If a new credential appears in Cloud Logging, the rotation has simply moved the exposure from Secret Manager to a log store that may have far broader access. Log only a masked representation, for example the first two characters and the value length, to confirm retrieval without exposing the credential.
Common mistakes
Rotating in Secret Manager but not in the source system. Adding a new secret version stores a new value, but if you did not actually generate a new credential at the database or API provider, the old and new versions contain the same value. Rotation achieved nothing. Always generate the new credential at the source before storing it.
Disabling the old version before consumers have migrated. Applications still using the old version fail authentication immediately when it is disabled. Use audit logs to confirm every caller has moved before you proceed. The overlap window exists precisely to prevent this.
Assuming applications pick up the latest version automatically. They do not always. Applications that read secrets at startup and cache the value in memory keep using the old value until they restart. Know each consumer’s behaviour before you proceed to Phase 2.
Logging the secret value in the rotation script. A single
console.log(secret)orprint(value)in a rotation function exposes the credential to anyone with log read access. Use masked output or omit the value entirely.No tested rollback procedure. If rotation fails mid-way, the new credential was added but health checks failed, you need to roll back to the previous version without causing an outage. The overlap window makes this possible, but only if you have a documented and tested rollback procedure ready before automation runs in production.
Keeping old versions enabled indefinitely. Old versions that are never disabled or destroyed extend the attack surface. Once all consumers have migrated and a rollback window has passed, disable then destroy old versions on a fixed schedule.
Best practices
Keep the overlap window short but safe. Long overlap windows mean two valid credentials exist for longer, increasing risk if either leaks. Short windows increase the chance of an outage if a consumer has not migrated in time. Tune the window to match the slowest consumer in your system.
Test rollback before enabling automation. Run a manual rotation end-to-end, simulate a health check failure, and verify you can roll back cleanly. Only then automate it.
Separate trigger logic from credential generation logic. The Pub/Sub handler should orchestrate by calling dedicated functions or services that handle credential generation, storage, and validation. This makes each step independently testable and easier to debug when something goes wrong.
Use least privilege for the rotation service account. The rotation handler only needs permission to add versions to specific secrets and to call the specific downstream system it manages. See
Why Service Account Keys Are Dangerous
for context on keeping service account scope tight.
Do not hardcode version numbers in consuming applications. Always use
latestor a symbolic alias rather than pinning to a specific version number. Pinned versions mean rotation has no effect until each consuming application is manually updated.
If your rotation handler fails silently, the secret will not rotate and you will not know. Emit a structured failure event from the handler and create a Cloud Monitoring alert on it so failures surface immediately rather than being discovered during the next incident.
Summary
- Secret Manager sends a Pub/Sub notification when the rotation window arrives; it does not generate new credentials by itself
- Your rotation handler (Cloud Function or Cloud Run) receives the Pub/Sub event and runs the actual rotation logic
- Rotation is a two-phase workflow: add the new version first, then propagate and retire the old one only after health checks pass
- Run both old and new credentials simultaneously during the overlap window; retire the old only after all consumers have migrated
- Not all applications pick up new secret versions automatically; know each consumer’s behaviour before disabling the old version
- Use Cloud Audit Logs (Data Access) to confirm which version each caller is reading before you disable the old version
- Never log secret values in rotation scripts; use masked output for debugging
- Test rollback procedures before enabling automated rotation in production
Frequently asked questions
What is automatic secret rotation in GCP?
Automatic secret rotation means replacing a credential on a schedule without manual intervention. In GCP, you configure a rotation period on a secret in Secret Manager. When the rotation window arrives, Secret Manager publishes a notification to a Pub/Sub topic. A Cloud Function or Cloud Run service receives that notification and runs your rotation logic: generating a new credential, storing it as a new secret version, updating consumers, and eventually retiring the old version. The trigger is automated; the rotation logic is code you write.
Does Secret Manager rotate secrets by itself?
No. Secret Manager sends a rotation notification to a Pub/Sub topic when the rotation window arrives, but it does not generate new credentials or update downstream systems on its own. You must write a rotation handler — a Cloud Function or Cloud Run service — that receives the Pub/Sub event, creates a new credential in the external system, stores it as a new version, and manages the rollout. Secret Manager handles the schedule and notification; your code does the actual rotation work.
What services can run the rotation logic in GCP?
Cloud Functions and Cloud Run are the most common options. Cloud Functions is a straightforward choice for simple rotation scripts — it triggers directly from Pub/Sub and requires minimal setup. Cloud Run gives you more control over execution environment, dependencies, and timeout limits, which matters for more complex rotation workflows. Both need a service account with least-privilege access to Secret Manager and to the downstream system being rotated.
Will applications automatically pick up the new secret version after rotation?
It depends on how each application reads secrets. Applications that reference the latest alias in Secret Manager will pick up the new version on their next API call. Applications that read secrets only at startup and cache the value in memory continue using the old credential until they restart. Cloud Run services that mount secrets as environment variables need redeployment; those that mount secrets as volume files receive updates automatically once the new version is latest. Always know each consumer's behaviour before you disable the old version.
How often should I rotate secrets?
For database passwords and service credentials, every 30 to 90 days is a common baseline. Compliance frameworks such as PCI-DSS and SOC 2 may mandate specific rotation intervals for certain credential types. The right interval depends on the sensitivity of the credential, the blast radius if it leaks, and whether your rotation automation is reliable enough to handle the frequency without disruption. Start with a longer interval, test thoroughly, then tighten it once you are confident in the automation.