Detecting Suspicious Activity in GCP Logs for Beginners

By the end of this page you will understand

• What suspicious activity looks like in GCP logs
• Which specific events are highest priority to detect
• How to write log queries to find those events
• How to build real-time alerts using Cloud Monitoring
• How logs, metrics, and alerts relate to each other
• What a basic response plan looks like when an alert fires

Simple explanation

Suspicious activity in GCP usually means one of two things: an identity is doing something it does not normally do, or someone is trying to gain access they should not have.

GCP keeps a detailed record of every API call made to its services. These records are called Cloud Audit Logs. When a user grants themselves new permissions, creates a service account key, or reads a secret, a log entry is written.

The problem is that logs do not react. They just record. If you look at your logs every morning, you might catch yesterday’s attack. If nobody looks, you might not notice for weeks. That gap between an event happening and someone acting on it is what detection and alerting closes.

The goal of this page is simple: configure your environment to alert you the moment a high-risk event happens, rather than discovering it after the fact. The events that matter most involve IAM changes, service account keys, impersonation, destructive operations, and unusual access patterns.

Why this matters

GCP environments are primarily attacked through credential theft and privilege escalation. An attacker who obtains a service account key or impersonates a privileged identity will make API calls that look identical to legitimate activity, unless you are specifically watching for the patterns that betray the intent.

IAM and credential events are the highest-value detection category in GCP. A new IAM binding granted at 3am from an unusual IP, a service account key created by an identity that has never done so before, or a bulk read of secrets: these events rarely occur during normal operations and frequently appear in the early stages of an attack. All of these signals come from Cloud Audit Logs, specifically Admin Activity and Data Access log entries.

Understanding the types of logs GCP generates and which ones are enabled by default matters before you build detections. Admin Activity logs are always on. Data Access logs capture reads and queries, but they must be explicitly enabled per service. They are off by default.

Without alerting, an attacker can move through your environment and you will only discover it during a routine review, if at all. The principle of least privilege limits what an attacker can do after they gain access, but detection and alerting tells you they were ever there.

How suspicious activity detection works in GCP

Detection in GCP is not a single product. It is a pipeline that connects several services together. Here is how it works end to end:

1. An action is performed

A user, service account, or external actor makes an API call. This could be granting an IAM role, creating a credential, accessing a secret, or deleting a resource.

2. Cloud Audit Logs records it

GCP writes a structured log entry to Cloud Audit Logs. The entry includes who performed the action, what they did, on which resource, from which IP address, and whether it succeeded.

3. Cloud Logging stores the entry

The log entry is ingested into Cloud Logging and becomes queryable in Logs Explorer. It is stored in your project’s default log bucket, or in a centralised sink if you have configured one.

4. A log-based metric matches the event

If you have created a log-based metric with a filter that matches the event — for example, any entry with methodName=“SetIamPolicy” — Cloud Logging increments that metric.

5. Cloud Monitoring fires an alert

An alerting policy in Cloud Monitoring watches the metric. When the value exceeds your configured threshold, it triggers a notification. You can create alerts that notify via email, PagerDuty, Pub/Sub, or Slack.

6. Responders investigate and contain

A team member receives the alert, investigates the log entry, determines whether it is a real incident, and follows a predefined response playbook. If confirmed, they contain the issue by revoking the IAM binding, disabling the key, or using VPC Service Controls to restrict further access.

High-value events to detect

Start with events that are rare in legitimate operations but common in attack scenarios. These are the highest signal-to-noise detections in a typical GCP environment:

IAM changes on production projects

Any SetIamPolicy call that adds a new binding, especially one that grants a primitive role (roles/owner, roles/editor) or grants any role to an external identity, should trigger an alert. Legitimate IAM changes in production are infrequent and should go through a controlled change process, not be applied directly. When this event fires unexpectedly, it can indicate privilege escalation by an attacker who has already gained a foothold.

Service account key creation

The method google.iam.admin.v1.CreateServiceAccountKey means someone created a long-lived credential. In environments that have adopted keyless authentication, this event should almost never occur. Any key creation in production is a signal worth investigating immediately. It may indicate an attacker creating a backdoor credential they can use even after the original access is revoked.

Service account impersonation

GenerateAccessToken calls indicate that one identity is impersonating a service account to obtain a short-lived token. When a high-privilege service account is impersonated from an unexpected source identity (especially an account that has never done this before), it can indicate an attacker using service account impersonation to elevate their access. Accounts with roles/owner or roles/iam.securityAdmin are the highest priority to monitor.

IAM and permission enumeration

Calls to GetIamPolicy, ListServiceAccounts, and TestIamPermissions from identities that do not normally perform these operations can indicate an attacker mapping your environment’s permission structure after initial access. This is reconnaissance: the attacker is learning what they can reach before moving further. These calls are usually harmless in isolation. The real signal is volume and timing: many calls in a short window from an unfamiliar identity.

Destructive operations on critical resources

storage.buckets.delete and bigquery.datasets.delete on production resources are irreversible. They may indicate an insider threat, a ransomware-style attack, or an accidental action that requires immediate intervention. Similarly, deletion of log sinks or disabling of audit logging should be treated as a high-severity event: these actions can blind your detection capability entirely.

Permission denied probing

PERMISSION_DENIED responses (gRPC status code 7) mean an identity tried to perform an action and was refused. A small number of these is normal. A burst of them from a single identity across multiple services is a sign that something is probing what it can access. This pattern often appears in the early stages of an attack when an attacker is testing the boundaries of a compromised credential.

Example detection queries

The queries below are starting points for investigation and tuning, not a complete managed detection programme. Run them in Logs Explorer or via the gcloud CLI to verify they return the expected results in your environment before building alerts on top of them.

# Detect IAM policy changes (SetIamPolicy) on a project
gcloud logging read \
  'logName="projects/my-app-prod/logs/cloudaudit.googleapis.com%2Factivity"
  AND protoPayload.methodName="SetIamPolicy"' \
  --project=my-app-prod \
  --limit=20 \
  --format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.resourceName)'

# Detect service account key creation
gcloud logging read \
  'protoPayload.methodName="google.iam.admin.v1.CreateServiceAccountKey"' \
  --project=my-app-prod \
  --limit=20 \
  --format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.resourceName)'

# Detect service account token generation (impersonation)
gcloud logging read \
  'protoPayload.methodName="GenerateAccessToken"' \
  --project=my-app-prod \
  --limit=20 \
  --format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.request.name)'

# Detect permission-denied events (PERMISSION_DENIED = gRPC code 7)
# Useful for spotting access probing by an attacker testing what they can reach
gcloud logging read \
  'logName:"cloudaudit.googleapis.com"
  AND protoPayload.status.code=7' \
  --project=my-app-prod \
  --limit=50 \
  --format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.methodName)'

When reviewing results, pay attention to protoPayload.authenticationInfo.principalEmail (who performed the action), protoPayload.requestMetadata.callerIp (where it came from), and protoPayload.resourceName (what resource was targeted). These three fields are the core of almost every investigation.

Building log-based alerts

A gcloud query run manually provides investigation capability. An automated alert provides detection. Use Cloud Monitoring log-based alerts to trigger notifications in real time when a matching log entry appears.

Log-based alerts consist of three parts: a log filter (the same syntax as gcloud logging read), a notification channel (email, PagerDuty, Pub/Sub, Slack webhook), and a rate or count threshold. For critical events like key creation or owner binding grants, set the threshold to 1 so any single occurrence triggers an alert. Learn how to create alerting policies in Cloud Monitoring.

# Step 1: Create a log-based metric to track SetIamPolicy events
gcloud logging metrics create iam-policy-changes \
  --description="Count of SetIamPolicy calls on this project" \
  --log-filter='protoPayload.methodName="SetIamPolicy"' \
  --project=my-app-prod

# Step 2: After creating the metric, create an alerting policy in Cloud Monitoring
# that triggers when the metric value exceeds 0 within a 5-minute window

Log Analytics for historical investigation

For investigations that need to look across a longer time window or correlate across multiple projects, Cloud Logging’s Log Analytics feature allows SQL-based queries against log buckets. Log Analytics is available for log buckets with analytics enabled and is particularly useful when you need to understand what happened before an alert fired.

-- Find all IAM changes in the last 30 days, grouped by principal
SELECT
  proto_payload.audit_log.authentication_info.principal_email AS principal,
  COUNT(*) AS change_count,
  MIN(timestamp) AS first_seen,
  MAX(timestamp) AS last_seen
FROM `my-app-prod.global._Default._AllLogs`
WHERE
  log_id = "cloudaudit.googleapis.com/activity"
  AND proto_payload.audit_log.method_name = "SetIamPolicy"
  AND timestamp > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
GROUP BY principal
ORDER BY change_count DESC

This query surfaces which principals have been making IAM changes most frequently. It can reveal unusual activity patterns even when no individual event was flagged as suspicious on its own.

Cloud Audit Logs vs log-based metrics vs alerts

These three things are related but distinct, and beginners often confuse them. Here is how they fit together:

Cloud Audit Logs

Cloud Audit Logs are raw records of what happened. They exist whether you read them or not. They are the source of truth for every detection on this page. Admin Activity logs are always on. Data Access logs must be enabled per service.

Log-based metrics

A log-based metric converts matching log entries into a measurable signal. It counts how many times a log filter has matched over time. On its own, a metric does not notify anyone. It just tracks a number.

Alerting policies

An alerting policy in Cloud Monitoring watches a metric and fires a notification when the value crosses a threshold. This is the part that actually tells someone something happened. The alert is what turns a log entry into an actionable event.

Security Command Center

Security Command Center is a broader security posture platform that sits above all of this. It can automatically generate findings from Cloud Audit Logs, vulnerabilities, and misconfigurations without you writing individual detection queries. Event Threat Detection (an SCC Premium feature) runs pre-built detections against your logs in real time. SCC is complementary to manual log-based detection, not a replacement for understanding how the underlying logs work.

When to use this

Log-based detection is most valuable when you need specific, tuned alerts for events that matter to your environment. Consider applying these detections in the following situations:

• Production IAM changes — any environment where direct IAM changes are unusual and should always be reviewed
• Service account key creation in keyless environments — if your team has moved to Workload Identity or impersonation, a key creation event is almost always anomalous
• Unusual access to secrets or sensitive data — high-sensitivity projects where Secret Manager or BigQuery data should only be read by specific identities
• Impersonation of privileged service accounts — any environment where high-privilege service accounts exist and impersonation should be audited
• Destructive operations on storage, BigQuery, or infrastructure — production environments where accidental or malicious deletion would have severe consequences
• Repeated permission denied probing — environments with externally exposed APIs or shared identities that could be compromised and tested
• Multi-project or organisation-level monitoring — shared environments where attack paths may span across project boundaries

Detection requires a response playbook

An alert without a defined response process creates alert fatigue and eventual ignorance. For each detection you build, document:

• Who owns the alert — which team or individual is paged and responsible for initial triage
• Initial triage steps — what to check first to determine if it is a true positive (e.g., correlate with a change request ticket, check the source IP against known corporate ranges)
• Escalation path — who to involve if the initial review confirms a real incident
• Containment actions — which specific actions to take immediately (e.g., revoke the IAM binding, disable the service account key, isolate the VM). Once you have confirmed malicious activity, use VPC Service Controls to restrict access and limit further lateral movement.

An alert that triggers and sits unanswered is worse than no alert: it burns trust in the detection system and trains the team to ignore future alerts.