AWS CloudWatch Log-Based Metrics Explained (Metric Filter Examples)

Who this page is for

This page is for developers and ops engineers who want to monitor application behavior using data that already exists in their logs. If you are already sending logs to CloudWatch Logs and want to alert on specific events without modifying your application code, metric filters are the right tool.

By the end of this page you will understand how metric filters work, when they are the right choice, how to create one, how to alarm on the result, and what mistakes to avoid in production.

The simple explanation

Your application writes a continuous stream of log lines. A metric filter sits beside that stream holding one rule. Every time a new line arrives, it checks: does this match? If yes, it records a count. Those counts become a CloudWatch metric you can chart and alarm on.

Your application does not change. The filter does the watching in the background.

How metric filters work

A metric filter connects three things: a log group, a pattern, and a metric transformation.

Log group

You attach the filter to one CloudWatch Logs log group. Every new log event written to that group is evaluated against your pattern. Each log group can have up to 100 metric filters.

Filter pattern

The pattern tells CloudWatch what to look for. It can be a plain text string, a JSON field comparison, or a space-delimited field match for fixed-format logs. See the filter pattern section below for examples of each format.

Metric transformation

When a log event matches, CloudWatch publishes a value to a metric you name. That metric lives in a namespace you choose, for example CustomApp/Payments. You define what value to publish: a constant 1 to count matches, or the value of a specific JSON field to track numbers like latency or order amounts.

The resulting metric works exactly like any native CloudWatch metric. You can graph it, set alarms on it, and include it in dashboards.

What happens after a match

CloudWatch publishes one data point per matching log event, not one per time period. If 50 log events match in a single minute, CloudWatch receives 50 data points. Your alarm and dashboard aggregations (sum, average, p99) work across those data points within whatever evaluation period you configure.

Historical logs are not included

When to use log-based metrics

Metric filters are a good fit when:

• You already log the event. No application code changes needed. If the data is in the logs, a filter costs you only a few minutes of configuration.
• You want continuous alerting on a known pattern. Metric filters run on every incoming event in real time. Once created, they run automatically with no query to schedule.
• The pattern is stable and well-defined. Login failures, specific error codes, high-latency responses, order completions. Things you know you will want to watch for months.
• You want monitoring on code you cannot change. Legacy services, third-party integrations, or vendor-managed applications. If it logs, you can filter.

Common use cases include failed login counts, payment errors, Lambda cold starts (extracted from the REPORT log line), HTTP 5xx error rates, specific exception types, and business events like subscription upgrades or refunds. For monitoring Lambda functions, metric filters on the function’s log group are a practical first layer of custom alerting.

When not to use log-based metrics

• Ad hoc investigation. Metric filters are not query tools. For debugging production issues or exploring unfamiliar failure patterns, use CloudWatch Logs Insights instead.

• Historical analysis. Metric filters have no data before they were created. To understand what happened in your logs last Tuesday, query the logs directly.

• High-cardinality metrics. If you want a separate metric per user, per request, or per any other high-volume dimension, the cost will grow quickly. Each unique dimension combination is a separate billable custom metric.

• Complex multi-line logic. Metric filters match individual log events. They cannot join events, compute moving averages, or run logic across multiple log lines. For that, process logs with a Lambda subscription filter.

• When your application already publishes the metric. If you use the AWS SDK or Embedded Metric Format (EMF) to publish the same measurement from code, adding a metric filter on top is redundant.

Filter pattern syntax

CloudWatch supports three pattern types. Choosing the right one matters for accuracy, especially if you use structured JSON logging.

Simple text match

CloudWatch checks whether the pattern string appears anywhere in the raw log event text. The match is case-sensitive. A pattern of ERROR matches any line containing the word ERROR, including JSON strings like “description”:“no error found” if the letters happen to appear. Use this only on plain-text logs.

JSON field match

For structured JSON logs, match on specific field values using dollar-sign dot notation. Example: { $.level = “ERROR” } matches only events where the top-level level field equals ERROR exactly. Nested fields use dot notation: { $.http.status = 500 }. Conditions combine with && and ||.

JSON matching is more reliable than text matching on structured logs and is what you should default to if your logs are JSON.

Space-delimited field patterns

For fixed-format text logs like Apache or nginx access logs, name each field by position. Example: [ip, -, user, timestamp, request, status=5*, size] matches lines where the sixth field starts with 5, catching HTTP 5xx errors. This format is less common now that most teams use structured logging.

How to create a metric filter

In the AWS Console

1. Open CloudWatch, navigate to Logs, then Log groups.
2. Select the log group you want to monitor.
3. Choose the Metric filters tab, then Create metric filter.
4. Enter your filter pattern. Use the Test pattern section to paste sample events and confirm the match is correct.
5. Choose Next and fill in: metric namespace, metric name, metric value, and default value.
6. Review and choose Create metric filter.

The filter is active immediately. New log events that match will appear in the metric within a few minutes of being logged.

With the AWS CLI

aws logs put-metric-filter \
  --log-group-name "/aws/lambda/payment-processor" \
  --filter-name "payment-errors" \
  --filter-pattern '{ $.level = "ERROR" }' \
  --metric-transformations \
    metricName=PaymentErrors,\
    metricNamespace=CustomApp/PaymentProcessor,\
    metricValue=1,\
    defaultValue=0

Naming conventions

Pick names you will recognize months later when reviewing dashboards and alarms.

• Namespace. Use a path like CustomApp/ServiceName. Group metrics by service. Avoid generic names like Custom that become unreadable once you have many services.
• Metric name. Use PascalCase and be specific. PaymentErrors beats Errors. CheckoutLatencyMs beats Latency.
• Filter name. Describe what it detects. payment-card-declined beats filter1.

Example: count errors from application logs

Suppose your Lambda function logs {“level”:“ERROR”,“message”:“Payment failed”,“code”:“CARD_DECLINED”} when a payment fails. You want to count these as a metric over time.

# Count ERROR log events in a Lambda log group
aws logs put-metric-filter \
  --log-group-name "/aws/lambda/payment-processor" \
  --filter-name "payment-errors" \
  --filter-pattern '{ $.level = "ERROR" }' \
  --metric-transformations \
    metricName=PaymentErrors,\
    metricNamespace=CustomApp/PaymentProcessor,\
    metricValue=1,\
    defaultValue=0

# Confirm the filter was created
aws logs describe-metric-filters \
  --log-group-name "/aws/lambda/payment-processor"

The defaultValue=0 ensures that during quiet periods CloudWatch publishes a zero rather than leaving a gap in the metric series. Without it, an alarm on this metric sits in INSUFFICIENT_DATA state when there are genuinely no errors, making it unreliable.

Example: extract latency from JSON logs

If your application logs response times as a JSON field, you can extract those actual values rather than just counting events. Suppose your API logs: {“endpoint”:“/checkout”,“duration_ms”:342,“status”:200}

# Extract duration_ms as a latency metric
aws logs put-metric-filter \
  --log-group-name "/aws/lambda/api-handler" \
  --filter-name "checkout-latency" \
  --filter-pattern '{ $.endpoint = "/checkout" && $.status = 200 }' \
  --metric-transformations \
    metricName=CheckoutLatencyMs,\
    metricNamespace=CustomApp/API,\
    metricValue='$.duration_ms',\
    unit=Milliseconds

With metricValue=‘$.duration_ms’, CloudWatch publishes the numeric value from the log field rather than a constant 1. You can then compute p50, p95, and p99 percentiles on this metric, alarm if p99 exceeds a threshold, and chart it alongside native AWS metrics on a CloudWatch dashboard.

How to create an alarm from a log-based metric

Once your metric filter is publishing data, creating an alarm on it works identically to creating an alarm on any native CloudWatch metric. Reference the same namespace and metric name you used in the filter. For a full walkthrough of alarm options and SNS notification setup, see the Creating Alerts guide.

# Alarm if payment errors exceed 10 in a 5-minute window
aws cloudwatch put-metric-alarm \
  --alarm-name "payment-errors-high" \
  --alarm-description "More than 10 payment errors in 5 minutes" \
  --namespace "CustomApp/PaymentProcessor" \
  --metric-name "PaymentErrors" \
  --statistic Sum \
  --period 300 \
  --threshold 10 \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 1 \
  --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:on-call-team

Key alarm fields explained for beginners:

• period. The time window in seconds over which data points are aggregated. 300 means a 5-minute window.
• statistic. How to aggregate data points in the period. Sum counts total events. Use Sum for error counts and percentiles for latency.
• threshold. The value the metric must exceed for the alarm to trigger.
• evaluation-periods. How many consecutive periods must breach the threshold before the alarm fires. Setting 1 triggers on the first breach.
• treat-missing-data. What to do when there are no data points in a period. notBreaching treats silence as OK. Use this together with defaultValue=0 on the metric filter for predictable alarm behavior.

Limits, pricing, and gotchas

Quotas

• Up to 100 metric filters per log group (soft limit, can be increased via a service quota request)
• Filter patterns cannot exceed 1,024 characters
• Each metric filter supports up to 3 metric transformations

Pricing

Each metric filter produces one or more CloudWatch custom metrics, billed at approximately $0.30 per metric per month after the first 10 free custom metrics in your account. This is the same rate as a custom metric published directly from application code.

Default value and data gaps

Without a defaultValue, CloudWatch publishes no data point during periods where no log events match. Alarms treat these gaps as INSUFFICIENT_DATA. For count metrics, set defaultValue=0 so quiet periods publish a zero, and pair this with treat-missing-data notBreaching on the alarm.

Metric data is separate from log data

The metric lives in CloudWatch Metrics, not in CloudWatch Logs. The original log events remain in your log group subject to your retention settings. You pay for log storage and for the custom metric separately.

Best practices

• Use structured JSON logging. JSON patterns match on field values, not arbitrary substrings. If your logs are still plain text, consider migrating to structured logging before building metric filters on top.

• Always set defaultValue=0 for count metrics. Without it, alarms behave unpredictably during periods with no matching events. Set defaultValue=0 so the metric has a continuous series, and pair it with treat-missing-data notBreaching on the alarm.

• Test the pattern with real sample events before saving. Use the Console’s test step with actual log events from your group. A pattern that looks correct can match events you did not expect.

• Keep namespaces and metric names consistent. Define a naming convention before you have 20 metrics. CustomApp/ServiceName for namespace and PascalCase for metric names keeps dashboards readable over time.

• Avoid high-cardinality dimensions. Do not use user IDs, request IDs, or any value that takes thousands of distinct values as a dimension. Each unique combination is a separate billable custom metric.

• Be deliberate about missing-data behavior on alarms. The AWS default is missing, which treats periods with no data as unknown. For count-based error metrics with defaultValue=0, set notBreaching so silence is treated as healthy.

• Audit and remove filters for retired features. Filters with defaultValue=0 that no longer match real events still generate empty metrics and count toward your custom metric total.

Metric filters vs Logs Insights vs custom metrics (EMF)

CloudWatch gives you several ways to get numbers out of your application. Here is how to choose between them.

These approaches are complementary. A common setup: metric filters for ongoing alerting on known patterns, Logs Insights for post-incident investigation, and EMF for precision metrics in critical paths. For how these tools work together during an incident, see the incident response with monitoring guide.