Event Hub Message Delivery Failures

Azure Event Hubs delivery failures fall into two categories: producer failures where messages never reach the hub, and consumer failures where messages arrive but processing breaks down. This page covers the real error messages for each failure type, how to read Event Hubs metrics to identify throttling and errors, and how to design reliable retry and error handling.

Producer failures: messages not arriving

Producer failures prevent messages from reaching the Event Hub. There are three distinct failure categories.

Authentication failure — the producer cannot connect at all:

Microsoft.Azure.EventHubs.EventHubsException:
Put token failed. status-code: 401, status-description: ExpiredToken:
The token is expired. Expiry time: ..., Current time: ...

For Shared Access Signature authentication, verify the connection string format and that the key has not been rotated:

# List Event Hub namespace authorization rules
az eventhubs namespace authorization-rule list \
  --resource-group myResourceGroup \
  --namespace-name myeventhubns \
  --output table

# Get the primary connection string
az eventhubs namespace authorization-rule keys list \
  --resource-group myResourceGroup \
  --namespace-name myeventhubns \
  --name RootManageSharedAccessKey \
  --query "primaryConnectionString" -o tsv

For managed identity authentication, verify the identity has the Event Hubs Data Sender role:

NAMESPACE_ID=$(az eventhubs namespace show \
  --resource-group myResourceGroup \
  --name myeventhubns \
  --query id -o tsv)

az role assignment create \
  --assignee "$MANAGED_IDENTITY_OBJECT_ID" \
  --role "Azure Event Hubs Data Sender" \
  --scope "$NAMESPACE_ID"

Throttling (ServerBusyException) — the producer is sending faster than the allocated throughput units allow:

Microsoft.Azure.EventHubs.ServerBusyException:
The server is busy. Please retry the operation.

Each throughput unit provides 1 MB/s or 1,000 events/s inbound. A Standard namespace supports up to 40 TUs. A Premium namespace uses Processing Units without hard TU limits.

Quota exceeded — namespace is at the limit for active connections or partition count:

Microsoft.Azure.EventHubs.QuotaExceededException:
The maximum number of allowed receivers per partition in a consumer group has been reached.

Reading Event Hubs metrics to identify throttling

The most useful metrics for diagnosing Event Hubs problems are:

  • ThrottledRequests: requests rejected due to exceeding throughput unit limits
  • ServerErrors: internal service errors (should be near zero in a healthy namespace)
  • UserErrors: client errors including authentication failures and quota violations
  • IncomingMessages / OutgoingMessages: message flow rates
  • ActiveConnections: current producer and consumer connections

View throttled request metrics over the last hour:

NAMESPACE_ID=$(az eventhubs namespace show \
  --resource-group myResourceGroup \
  --name myeventhubns \
  --query id -o tsv)

az monitor metrics list \
  --resource "$NAMESPACE_ID" \
  --metric "ThrottledRequests" \
  --interval PT1M \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --aggregation Total \
  --output table

If ThrottledRequests is consistently above zero, add more throughput units:

az eventhubs namespace update \
  --resource-group myResourceGroup \
  --name myeventhubns \
  --capacity 5

Enable auto-inflate for automatic TU scaling (Standard tier only):

az eventhubs namespace update \
  --resource-group myResourceGroup \
  --name myeventhubns \
  --enable-auto-inflate true \
  --maximum-throughput-units 20
Note

Auto-inflate scales up TUs automatically when throttling occurs, but it does not scale down. You must manually reduce TUs when load decreases to avoid unnecessary cost. Premium and Dedicated tiers use a different capacity model and do not support auto-inflate.

Consumer failures: messages not being processed

Offset out of range — the consumer has a checkpoint older than the retention period:

Microsoft.Azure.EventHubs.EventHubsException:
The supplied offset '1234567' is past the end of the stream for
consumer group '$Default' and EventHub 'myeventhub'. The requested
offset is 1234567, while the last offset in the system is 789012.

This happens when a consumer is offline for longer than the retention period (default 1 day, up to 7 days on Standard, 90 days on Premium). The checkpoint offset no longer exists in the event log.

Reset to the latest available offset in the Python SDK:

from azure.eventhub import EventHubConsumerClient, EventPosition

client = EventHubConsumerClient.from_connection_string(
    conn_str=connection_string,
    consumer_group="$Default",
    eventhub_name="myeventhub"
)

# Start from the latest offset (skip missed messages)
client.receive(
    on_event=process_event,
    starting_position=EventPosition("@latest"),
)

# Or start from the earliest available data
client.receive(
    on_event=process_event,
    starting_position=EventPosition("-1"),
)

Consumer group conflict — two independent consumer applications sharing the same group:

Microsoft.Azure.EventHubs.ReceiverDisconnectedException:
New receiver with higher epoch of '0' is created hence current receiver
with epoch '0' is getting disconnected.

Event Hubs allows only one active receiver per partition per consumer group. Fix by creating a dedicated consumer group for each consumer application:

az eventhubs eventhub consumer-group create \
  --resource-group myResourceGroup \
  --namespace-name myeventhubns \
  --eventhub-name myeventhub \
  --name myapp2-consumer-group

Checkpoint failure — the consumer cannot save its read position to Blob Storage:

BlobServiceClient is not configured. Checkpointing cannot proceed.

Checkpointing requires write access to an Azure Blob Storage container. Grant the managed identity Storage Blob Data Contributor on the checkpoint container:

CHECKPOINT_SCOPE="/subscriptions/$SUB_ID/resourceGroups/$RG/providers/Microsoft.Storage/storageAccounts/mystorageaccount/blobServices/default/containers/checkpoints"

az role assignment create \
  --assignee "$MANAGED_IDENTITY_OBJECT_ID" \
  --role "Storage Blob Data Contributor" \
  --scope "$CHECKPOINT_SCOPE"

Schema Registry mismatch errors

When using Azure Event Hubs Schema Registry with Avro or JSON schemas, producers and consumers must agree on the schema version. A mismatch causes deserialization failures on the consumer:

System.Exception: Schema not found. Schema group: 'mygroup', Schema name: 'com.example.MyEvent'

Check registered schemas:

az eventhubs namespace schema-registry list \
  --resource-group myResourceGroup \
  --namespace-name myeventhubns \
  --output table

Schema Registry supports three compatibility modes:

  • None: no compatibility check, any version accepted
  • Backward: new schema can read data written with the old schema
  • Forward: old schema can read data written with the new schema

If a producer deploys a new schema version before consumers are updated, and the compatibility mode is Forward, consumers will fail to deserialize new messages. Always deploy consumer updates before producer updates when making schema changes.

Tip

For production event streaming, deploy consumer updates before producer updates when making schema changes. This ensures consumers can handle both old and new message formats during the transition window before all producers are updated.

Handling failed messages: the dead-letter pattern

Event Hubs has no native dead-letter queue. Failed processing must be handled in application code. The standard pattern writes failed messages to a separate Blob Storage container or a secondary Event Hub:

async def process_event(partition_context, event):
    try:
        await handle_business_logic(event)
        await partition_context.update_checkpoint(event)
    except Exception as e:
        # Write to dead-letter store before checkpointing
        failed_record = {
            "body": event.body_as_str(),
            "sequence_number": event.sequence_number,
            "partition_id": partition_context.partition_id,
            "error": str(e),
            "failed_at": datetime.utcnow().isoformat()
        }
        await dead_letter_container.upload_blob(
            name=f"failed/{partition_context.partition_id}/{event.sequence_number}.json",
            data=json.dumps(failed_record),
            overwrite=True
        )
        # Still checkpoint to prevent infinite retry loop
        await partition_context.update_checkpoint(event)

For Event Hubs triggered Azure Functions, the Functions runtime retries a batch of events up to the configured maxDequeueCount before moving to the next batch. Configure the retry behavior in host.json:

{
  "extensions": {
    "eventHubs": {
      "batchCheckpointFrequency": 1,
      "eventProcessorOptions": {
        "maxBatchSize": 100,
        "prefetchCount": 300
      }
    }
  }
}

If dead-letter handling is a hard requirement — with automatic retry, message lock renewal, and session support — consider Azure Service Bus Topics, which provide native dead-letter queues at the cost of lower throughput than Event Hubs.

Common mistakes

  1. Sharing a consumer group between two applications. Event Hubs allows only one active receiver per partition per consumer group. Two applications sharing a group cause ReceiverDisconnectedException errors and message processing gaps. Always create a dedicated consumer group for each independent consumer application.
  2. Not implementing retry logic in producers for ServerBusyException. Throttled sends return ServerBusyException and the SDK does not automatically retry in all configurations. Without exponential backoff retry logic in the producer, messages are silently dropped during throttling events. Configure the SDK retry policy explicitly with maximum retry count and backoff intervals.
  3. Assuming checkpoints survive storage account deletion. Checkpoints are stored externally in Blob Storage. If that storage account is deleted or the container is cleared, consumers restart from the beginning of the retention window or the latest offset. Protect the checkpoint storage account with resource locks and regular backups.
  4. Setting retention to 1 day with occasional multi-day consumer downtime. If a consumer is offline longer than the retention period, its checkpoints become invalid and it will miss messages that expired from the log. Set retention to at least twice the maximum expected consumer downtime window, especially for batch processing jobs that run weekly.

Related topics to read next

Event Hubs failures connect to identity, monitoring, and storage topics that underpin reliable event streaming.

  • Azure Monitor Overview — create metric alerts on ThrottledRequests and ServerErrors to detect Event Hubs health problems automatically before producers start dropping messages.
  • Managed Identities Overview — use managed identity instead of SAS keys to authenticate producers and consumers to Event Hubs without rotating secrets.
  • Blob Storage Security — secure the checkpoint store that Event Hubs consumers use to track their read position across restarts.

Frequently asked questions

What happens to messages when Event Hubs throttles a producer?

Throttled send operations return a ServerBusyException (HTTP 503) or a QuotaExceededException. Messages are not automatically retried by the service — the producer SDK must implement retry logic with exponential backoff. Messages that are not retried are lost. Event Hubs does not have a built-in dead-letter queue for throttled sends.

Does Azure Event Hubs have a dead-letter queue?

No. Unlike Azure Service Bus, Event Hubs does not have a native dead-letter queue. Failed consumer processing must be handled in application code — typically by writing failed messages to a separate storage location. Use Azure Service Bus if you need native dead-lettering.

Can two consumer applications share a consumer group in Event Hubs?

No. Each consumer group is designed for exactly one consumer application. Two competing consumers using the same consumer group will fight over partition ownership, causing ReceiverDisconnectedException errors and unreliable message delivery. Create a separate consumer group for each independent consumer application.

Last verified: 19 March 2026 Cloud services change frequently. Verify details against official documentation before making infrastructure decisions.