Event Driven Systems in Azure

Core concepts of event-driven design

An event is an immutable record of something that happened in the past. “Order #12345 was placed at 14:32 UTC for customer #67890” is an event. Events are facts — they describe what occurred, not what should happen next. The producer has no obligation to know who is interested in the event or what they will do with it.

A command is different from an event. A command is a request for something to happen — “charge the customer’s card.” Commands are directed at a specific recipient and imply a response. Events are broadcast without expecting a direct response.

A message broker stores events temporarily and delivers them to interested consumers. The broker handles durability (events are not lost if the consumer is down), delivery guarantees, and routing. Azure’s three messaging services offer different points in the design space of throughput, ordering guarantees, delivery semantics, and consumer patterns.

Azure Service Bus for business workflows

Azure Service Bus is a fully managed enterprise message broker. It supports two patterns: queues (point-to-point — one sender, one receiver) and topics with subscriptions (publish-subscribe — one sender, many receivers). Messages in Service Bus are delivered at least once and can be configured for ordered (FIFO) delivery using message sessions.

Key Service Bus features for reliable messaging include:

• Dead-letter queue (DLQ): Messages that cannot be processed (exceeded delivery count, TTL expired, or explicitly dead-lettered by the consumer) are moved to the DLQ rather than being lost. Monitor the DLQ and alert on non-empty state.
• Message sessions: Group related messages under a session ID and guarantee that a single consumer processes all messages in a session in order. Useful for order processing where events for a specific order must be processed sequentially.
• Deferred messages: A consumer can defer a message it is not ready to process, leaving it in the queue to be fetched later by sequence number.
• Scheduled messages: Publish a message now but make it available to consumers only after a specified time.

# Create a Service Bus namespace and topic with subscription
az servicebus namespace create \
  --resource-group myRG \
  --name mysbns \
  --sku Standard \
  --location eastus

az servicebus topic create \
  --resource-group myRG \
  --namespace-name mysbns \
  --name order-events \
  --default-message-time-to-live P14D \
  --max-size-in-megabytes 5120

az servicebus topic subscription create \
  --resource-group myRG \
  --namespace-name mysbns \
  --topic-name order-events \
  --name inventory-processor \
  --max-delivery-count 5 \
  --dead-lettering-on-message-expiration true \
  --lock-duration PT1M

Azure Event Hubs for high-throughput streaming

Event Hubs is designed for ingesting millions of events per second from IoT devices, application telemetry, clickstream data, or any scenario where the volume far exceeds what a traditional message broker handles efficiently. Events are stored for a configurable retention period (1–90 days on Dedicated tier) and can be replayed, which is impossible with Service Bus queues.

Event Hubs organises events into partitions. Each partition is an ordered sequence of events. The number of partitions is fixed at creation time and determines the maximum parallelism for consumers. Events with the same partition key are always routed to the same partition, guaranteeing ordering within a partition key.

Event Hubs Capture automatically archives events to Azure Blob Storage or Azure Data Lake Storage in Avro format. This creates a permanent audit log without any consumer code and feeds downstream analytics with Azure Stream Analytics, Databricks, or Synapse Analytics.

# Create an Event Hubs namespace and event hub
az eventhubs namespace create \
  --resource-group myRG \
  --name myehns \
  --sku Standard \
  --location eastus

az eventhubs eventhub create \
  --resource-group myRG \
  --namespace-name myehns \
  --name device-telemetry \
  --partition-count 8 \
  --message-retention-in-days 7

# Enable Capture to Blob Storage
az eventhubs eventhub update \
  --resource-group myRG \
  --namespace-name myehns \
  --name device-telemetry \
  --enable-capture true \
  --capture-interval 300 \
  --capture-size-limit 314572800 \
  --destination-name EventHubArchive.AzureBlockBlob \
  --storage-account /subscriptions/SUB_ID/resourceGroups/myRG/providers/Microsoft.Storage/storageAccounts/myarchive \
  --blob-container telemetry-archive

Azure Event Grid for reactive notifications

Event Grid is an event routing service, not a message broker. It is designed for low-volume, discrete notification events — “a blob was created,” “a resource group was deleted,” “a custom business event occurred.” Event Grid delivers events to subscribers with very low latency (milliseconds) but does not retain events for replay.

Event Grid sources include every major Azure service (Blob Storage, App Service, Container Registry, resource group changes) as well as custom topics where your application publishes events. Subscribers can be Azure Functions, Logic Apps, Webhooks, Service Bus queues, or Event Hubs. This makes Event Grid ideal for triggering processing pipelines on resource events.

# Create an Event Grid subscription on a Blob Storage account
# (publishes an event to a Function every time a blob is created)
az eventgrid event-subscription create \
  --name blob-created-trigger \
  --source-resource-id /subscriptions/SUB_ID/resourceGroups/myRG/providers/Microsoft.Storage/storageAccounts/mystorageacct \
  --endpoint-type azurefunction \
  --endpoint /subscriptions/SUB_ID/resourceGroups/myRG/providers/Microsoft.Web/sites/myFunctionApp/functions/ProcessNewBlob \
  --included-event-types Microsoft.Storage.BlobCreated \
  --subject-begins-with /blobServices/default/containers/uploads/

Choosing the right messaging service

Reliability patterns

Outbox pattern: Write the domain event to an outbox table in the same database transaction as the business operation. A relay process reads from the outbox and publishes to Service Bus, deleting the row after confirmed delivery. This guarantees that every database write produces exactly one corresponding message, even if the publish step fails and must be retried.

Idempotent consumers: Service Bus delivers messages at least once. Design consumers to be idempotent — processing a message twice produces the same outcome as processing it once. Record processed message IDs in a durable store. Check on receipt; skip if already processed.

Saga pattern: Long-running business transactions that span multiple services use the saga pattern. Each service performs its local operation and publishes an event. Subsequent services listen for that event and perform their step. Compensating transactions handle rollback — if the payment step fails after inventory was already reserved, a compensating event releases the reservation.

Event sourcing: Instead of storing the current state of an entity, store every event that led to that state. The current state is derived by replaying events. Event Hubs and Cosmos DB change feed both support event sourcing patterns. This provides a complete audit trail and enables temporal queries (“what did the account balance look like at 14:00 yesterday?”).

Monitoring event-driven systems

The most important metrics to monitor in an event-driven system are queue depth and dead-letter queue depth. A growing queue depth means consumers cannot keep up with producers. A non-zero DLQ depth means messages are failing processing and require investigation.

# Check active message count and DLQ count for a Service Bus topic subscription
az servicebus topic subscription show \
  --resource-group myRG \
  --namespace-name mysbns \
  --topic-name order-events \
  --name inventory-processor \
  --query "{ActiveMessages:countDetails.activeMessageCount, DLQMessages:countDetails.deadLetterMessageCount}"

# Set up a metric alert when DLQ count > 0
az monitor metrics alert create \
  --resource-group myRG \
  --name dlq-alert \
  --scopes /subscriptions/SUB_ID/resourceGroups/myRG/providers/Microsoft.ServiceBus/namespaces/mysbns \
  --condition "avg DeadletteredMessages > 0" \
  --description "Dead letter queue has messages requiring investigation" \
  --evaluation-frequency PT5M \
  --window-size PT15M \
  --severity 2