Event-Driven Architecture in GCP: Pub/Sub, Eventarc, and When to Use Each

Simple explanation

In a traditional system, Service A calls Service B directly and waits for a response. If Service B is slow or down, Service A is stuck. If you later need Service C to react to the same action, you modify Service A to call Service C too.

In an event-driven system, Service A publishes an event: a small message describing what happened, like “order placed” or “file uploaded”. Service A does not call anyone. It publishes and moves on. Services B and C each subscribe to that event type and process it independently, at their own pace.

Four concepts matter here:

• Producer: the service that emits the event. It knows nothing about who consumes it.
• Event: a message describing something that happened. It carries data (order ID, file path, user ID) but no instructions about what to do with it.
• Consumer: a service that receives and reacts to events. Multiple consumers can independently process the same event.
• Decoupling: producers and consumers have no direct dependency on each other. You can add, remove, or update consumers without touching the producer.

How event-driven systems work in GCP

GCP has two main services for event-driven patterns, and they cover different use cases.

Pub/Sub: application events

Pub/Sub is a managed messaging service. Your application code publishes messages to a topic. Each subscription on that topic receives every message independently. If a topic has three subscriptions, each subscriber gets its own copy of every message. They do not share a queue.

The flow works like this:

1. Your code publishes a message to a Pub/Sub topic (e.g. order-events).
2. Pub/Sub durably stores the message and delivers it to every subscription on that topic.
3. Each subscriber processes the message at its own pace. If one subscriber is temporarily down, its messages queue in Pub/Sub until it recovers.
4. Each subscriber acknowledges (acks) the message after processing. If it does not ack in time, Pub/Sub redelivers.

Subscribers can receive messages via push delivery (Pub/Sub sends an HTTP POST to your endpoint) or pull delivery (your code polls Pub/Sub for new messages).

Eventarc: GCP infrastructure events

Eventarc captures events emitted by GCP services and routes them to Cloud Run or Cloud Functions. You do not write publishing code. Instead, you declare a trigger: “when this GCP event happens, call this service”.

Common Eventarc sources include:

• A file is created in Cloud Storage
• A GCP API call is recorded in Cloud Audit Logs
• A document changes in Firestore

The key difference

Pub/Sub is for events your application generates. You control the topic, message format, and publishing logic. Eventarc is for events GCP services generate. You configure a trigger and GCP handles the rest. Many production systems use both: Pub/Sub for application events between microservices, and Eventarc for reacting to infrastructure changes.

Real example: order processing with Pub/Sub

Here is what a typical order-placed event flow looks like in GCP:

User places order
       |
Checkout service (Cloud Run)
       |
  Publishes: order-events topic
  Message: { "order_id": "ord-8821", "user_id": "u-4432", "total": 49.99 }
  Returns: HTTP 202 Accepted to user
       |
       +---> fulfilment-order-sub (Fulfilment Service, Cloud Run)
       |         Picks the order, updates warehouse system
       |
       +---> email-order-sub (Email Service, Cloud Run)
       |         Sends order confirmation email to customer
       |
       +---> analytics-order-sub (Analytics Service)
                 Writes to BigQuery for reporting

Each subscriber receives a full copy of the message and processes it independently. The fulfilment service can be down for 10 minutes. Its messages queue in Pub/Sub and are processed when it recovers, with no impact on the email service or analytics pipeline. Adding a fourth subscriber (say, a fraud detection service) requires zero changes to the checkout service: just add a new subscription to the existing topic.

Setting up the topic and subscriptions

# Create the order events topic
gcloud pubsub topics create order-events --project=my-app-prod

# Create separate subscriptions for each consuming service
gcloud pubsub subscriptions create fulfilment-order-sub \
  --topic=order-events \
  --ack-deadline=60 \
  --project=my-app-prod

gcloud pubsub subscriptions create email-order-sub \
  --topic=order-events \
  --ack-deadline=60 \
  --project=my-app-prod

gcloud pubsub subscriptions create analytics-order-sub \
  --topic=order-events \
  --ack-deadline=60 \
  --project=my-app-prod

The —ack-deadline=60 sets how long Pub/Sub waits for an acknowledgement before redelivering. Set this to longer than your typical processing time. If processing takes 45 seconds and your ack deadline is 30, Pub/Sub will redeliver the message while you are still working on it, causing duplicate processing.

Real example: infrastructure event with Eventarc

Suppose you want to process user-uploaded files automatically. With Pub/Sub, you would need to write code in your upload handler to publish a message after saving the file. With Eventarc, you skip that step entirely. GCP detects the file landing in Cloud Storage and invokes your processing service directly.

# Trigger a Cloud Run service when a file lands in Cloud Storage
gcloud eventarc triggers create process-upload-trigger \
  --location=us-central1 \
  --destination-run-service=file-processor \
  --destination-run-region=us-central1 \
  --event-filters="type=google.cloud.storage.object.v1.finalized" \
  --event-filters="bucket=my-app-prod-uploads" \
  --service-account=eventarc-sa@my-app-prod.iam.gserviceaccount.com \
  --project=my-app-prod

When a file lands in my-app-prod-uploads, Eventarc invokes the file-processor Cloud Run service with the event payload in the request body. No polling loop, no custom publishing code, no background job checking for new files.

Eventarc also works with Cloud Functions triggers, which is a natural fit for lightweight processing like image thumbnailing or metadata extraction.

Pub/Sub vs Eventarc vs Cloud Tasks

These three GCP services handle asynchronous work, but they solve different problems. Choosing the wrong one leads to overengineered solutions or missing features you actually need. For a deeper comparison of Pub/Sub and Cloud Tasks specifically, see the Pub/Sub vs Cloud Tasks guide.

When to use event-driven architecture

Event-driven architecture adds complexity. It is worth that complexity in specific situations:

• Order processing and e-commerce flows. One order event triggers fulfilment, email, analytics, and inventory updates independently. No service waits for any other.
• Background email and notifications. The user-facing request returns immediately. The notification is sent asynchronously and can be retried if it fails.
• Audit and log processing. Security events, access logs, and compliance records are published and consumed by multiple systems (SIEM, analytics, alerting) without the source system knowing about any of them.
• File upload processing. A file lands in Cloud Storage and triggers thumbnail generation, virus scanning, metadata extraction, or Dataflow pipelines without the upload handler managing any of it.
• Analytics and event pipelines. Clickstream data, application metrics, and business events flow through Pub/Sub into BigQuery, Dataflow, or other downstream systems.
• Microservices fan-out. Multiple independently deployed services react to the same business event without coupling to each other. This is a core pattern in microservices architectures.

When not to use it

Not every async problem needs an event bus. These situations are better served by simpler patterns:

• Synchronous user-facing workflows. If the user needs a result before the page loads (price calculation, search results, login), use direct HTTP or gRPC. Adding Pub/Sub to a request-response flow just adds latency and complexity.
• Single-consumer controlled dispatch. If exactly one service needs to process a task, with rate limiting and scheduling, Cloud Tasks is the right tool. Pub/Sub’s fan-out model is unnecessary overhead here.
• Simple scheduled jobs. If you need to run a batch process every hour, use Cloud Scheduler to invoke a Cloud Run service or Cloud Function directly. No topic, no subscription, no event handler.
• Low-volume internal workflows. If Service A calls Service B five times a day and there are no other consumers, a direct HTTP call is simpler, faster, and easier to debug. Event-driven architecture adds value when you need decoupling or fan-out, not for every service interaction.

Production best practices

Idempotency

Pub/Sub delivers at-least-once. Under some conditions (subscriber timeout, network interruption, pod restart) the same message may be delivered more than once. Every event handler must be idempotent: processing the same message twice must produce the same outcome as processing it once.

Common approaches:

• Deduplication key. Before processing, check a database or Redis cache for the Pub/Sub message ID. If you have seen it, acknowledge and skip. Store processed IDs with a TTL equal to your message retention period.
• Naturally idempotent operations. Setting a field to a value is idempotent (set it twice, the result is the same). Incrementing a counter is not. Prefer operations that produce the same state regardless of how many times they run.
• Upsert instead of insert. Use INSERT OR UPDATE with the event ID as the primary key. If the message is processed twice, the second upsert overwrites the first with identical data.

Retries and ack deadlines

Set the ack deadline longer than your expected processing time. If processing takes 45 seconds and the deadline is 30, Pub/Sub redelivers while you are still working on the first copy. For long-running handlers, extend the deadline programmatically using modifyAckDeadline.

Use exponential backoff for retry delays to avoid hammering a recovering downstream service. Pub/Sub supports configurable retry policies on subscriptions.

Dead-letter topics

Some messages cannot be processed successfully: malformed JSON, a reference to a deleted record, a dependency that is consistently unavailable. Without a dead-letter topic, Pub/Sub retries these until the retention period expires, then drops them silently. For a deeper look at failure modes, see Pub/Sub Message Delivery Failures.

# Create a dead-letter topic for failed order messages
gcloud pubsub topics create order-events-dead-letter --project=my-app-prod

# Update the subscription to use the dead-letter topic
gcloud pubsub subscriptions update fulfilment-order-sub \
  --dead-letter-topic=order-events-dead-letter \
  --max-delivery-attempts=5 \
  --project=my-app-prod

Set up an alert on the dead-letter subscription’s undelivered message count. Messages accumulating there indicate a consistent processing problem that needs investigation, not just transient noise.

Ordering keys

Pub/Sub does not guarantee order by default. If you publish events A, B, C, a subscriber may receive B, A, C. For most workloads this is fine. Welcome emails and analytics events do not need sequencing.

For events representing state transitions (“order created”, then “payment received”, then “order shipped”), enable ordering on the subscription and set an ordering key on related messages. Use a natural entity identifier like order_id. Be aware this requires a regional endpoint and reduces throughput.

Observability and monitoring

Monitor these metrics in Cloud Monitoring for every production subscription:

• Oldest unacked message age. If this grows, your consumers are falling behind.
• Undelivered message count. Watch this especially on dead-letter subscriptions.
• Push request latency. For push subscriptions, high latency means slow processing.
• Acknowledgement rate. A drop indicates processing failures.

IAM and service accounts

Each publishing service and each subscribing service should use its own dedicated service account. Grant roles/pubsub.publisher only to services that publish, and roles/pubsub.subscriber only to services that consume. For Eventarc triggers, the trigger’s service account needs roles/run.invoker on the destination Cloud Run service.

Avoiding silent failure paths