Automated Testing in AWS CodeBuild: Buildspec, Reports and Coverage

What automated testing in CodeBuild means

CodeBuild is a managed build service. You define commands in a file called buildspec.yml and CodeBuild runs them inside a fresh container every time a build triggers. When your test command exits non-zero, which every major test runner does when tests fail, CodeBuild marks the build as failed and stops the pipeline.

That is the entire mechanism. There is no special test plugin or test API. If pytest exits 1 because a test failed, CodeBuild reports a failure. If it exits 0, the build continues. Test reports, coverage tracking, and parallel builds are all built on top of that simple foundation.

How automated testing in CodeBuild works

Every CodeBuild test run follows this sequence:

1. Source — CodeBuild pulls your code from GitHub, CodeCommit, S3, or another connected source.
2. Build container starts — CodeBuild launches a fresh container using the runtime you specified. Nothing carries over from previous builds.
3. Install phase — your dependencies are installed, or restored from cache if caching is configured.
4. Build phase — your test commands run. This is where tests live.
5. Pass or fail — if any command exits non-zero, the build phase fails and CodeBuild marks the overall build as failed. post_build still runs after a failed build phase, which is worth understanding before you put deployment commands there.
6. Reports and artifacts — CodeBuild uploads JUnit and coverage XML files defined in the reports section, and uploads build artifacts to S3.

If you are new to CodeBuild itself, the AWS CodeBuild Overview covers project configuration, environments, and the full buildspec format before you wire in tests.

When to use this

• Pull request validation — run unit tests on every PR so developers get fast feedback before code review.
• Deployment gates — require a passing test build before any deployment stage runs in CodePipeline.
• Coverage enforcement — use a coverage threshold flag to prevent test coverage from silently dropping over time.
• Integration tests before release — run tests against a staging database or service before deploying to production.
• Multi-language projects — use batch builds to run tests for frontend, backend, and infrastructure in parallel instead of sequentially.

Before you start

You need these in place before setting up automated tests in CodeBuild:

• A repository with tests already written and a test runner configured (pytest, Jest, go test, etc.).
• A buildspec.yml file at the root of your repository, or a custom path set in your CodeBuild project.
• The correct runtime version specified in the buildspec install phase.
• Test output files in a format CodeBuild understands: JUnit XML for test results, Cobertura XML for coverage.
• If running Docker-based integration tests: privileged mode enabled in the CodeBuild project environment settings.
• If connecting to private resources (RDS, ElastiCache): VPC configuration on the CodeBuild project, including subnets, security groups, and a NAT gateway or VPC endpoints for outbound internet access.
• Secrets stored in AWS Secrets Manager or SSM Parameter Store, not hardcoded in buildspec.yml.

Running unit tests in buildspec.yml

Unit tests belong in the build phase. When a command in the build phase exits non-zero, CodeBuild marks that phase as failed and stops remaining commands in that phase. The build is reported as failed to CodePipeline, which stops downstream stages from running.

If you need cleanup to run regardless of test outcome, use a finally block inside the phase. The finally commands run even when the commands in that phase fail.

Python with pytest

This buildspec runs pytest with coverage and fails the build if line coverage drops below 80%.

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.12
    commands:
      - pip install -r requirements.txt
      - pip install pytest pytest-cov

  build:
    commands:
      - pytest tests/unit/ --cov=src --cov-report=xml --cov-fail-under=80 -v

  post_build:
    commands:
      - echo "Build complete. Checking success before pushing image..."
      - |
        if [ $CODEBUILD_BUILD_SUCCEEDING -eq 1 ]; then
          docker build -t my-app:$CODEBUILD_RESOLVED_SOURCE_VERSION .
        fi

reports:
  pytest-coverage:
    files:
      - coverage.xml
    file-format: COBERTURAXML

The —cov-fail-under=80 flag tells pytest to exit non-zero if line coverage is below 80%. Start with a realistic number, even 50%, and raise it over time rather than skipping coverage enforcement altogether.

Node.js with Jest

This buildspec runs Jest in CI mode and generates a Cobertura coverage report.

version: 0.2

phases:
  install:
    runtime-versions:
      nodejs: 20
    commands:
      - npm ci

  build:
    commands:
      - npx jest --ci --coverage --coverageReporters=cobertura --coverageDirectory=./coverage

reports:
  jest-coverage:
    files:
      - coverage/cobertura-coverage.xml
    file-format: COBERTURAXML

The —ci flag makes Jest fail on snapshot mismatches rather than auto-updating them, which is important for consistent CI behavior.

Go

This buildspec runs all Go tests with verbose output and generates a coverage profile.

version: 0.2

phases:
  install:
    runtime-versions:
      golang: 1.22

  build:
    commands:
      - go test ./... -v -coverprofile=coverage.out
      - go tool cover -func=coverage.out

Publishing test results and coverage reports

CodeBuild can parse JUnit XML test reports and Cobertura XML coverage reports and display them in the console, showing you exactly which tests passed and failed without reading through raw log output. This is especially useful when a test suite has hundreds of tests.

Add a reports section to your buildspec and point it at the XML files your test runner generates. This example generates both a JUnit result file and a Cobertura coverage file from the same pytest run:

version: 0.2

phases:
  install:
    commands:
      - pip install pytest pytest-cov

  build:
    commands:
      - pytest tests/ --junitxml=test-results/results.xml --cov=src --cov-report=xml

reports:
  pytest-results:
    files:
      - test-results/results.xml
    file-format: JUNITXML
  pytest-coverage:
    files:
      - coverage.xml
    file-format: COBERTURAXML

After the build, open the CodeBuild project in the console, click the build run, and go to the Reports tab. CodeBuild shows a pass or fail summary per test with timestamps and test names. This integrates with CodePipeline so a failed stage shows which specific test caused the failure.

Running integration tests in CodeBuild

Unit tests run in isolation against mocks. Integration tests need real dependencies: a database, a queue, or an external service. CodeBuild supports two approaches. For a full picture of where integration tests fit into a deployment pipeline, see CI/CD Pipelines for ECS.

Option 1: Docker containers inside CodeBuild

CodeBuild supports Docker-in-Docker when you enable privileged mode on the project in its environment settings. You can start a Postgres container, wait for it to be ready, run tests against it, and clean up, all within one build. No additional AWS resources or VPC configuration needed.

With privileged mode enabled, use this buildspec:

version: 0.2

phases:
  pre_build:
    commands:
      - docker run -d --name test-db \
          -e POSTGRES_PASSWORD=testpass \
          -e POSTGRES_DB=testdb \
          -p 5432:5432 \
          postgres:16
      - until docker exec test-db pg_isready; do sleep 1; done

  build:
    commands:
      - export DATABASE_URL=postgresql://postgres:testpass@localhost:5432/testdb
      - pytest tests/integration/ -v
    finally:
      - docker stop test-db && docker rm test-db

The finally block inside the build phase cleans up the container even if tests fail. Each build starts from a fresh container, so there is no data leakage between runs.

If you also build Docker images as part of your pipeline, see Building Docker Images with CodeBuild for how image builds and test builds fit together.

Option 2: VPC access to a private RDS instance

For tests that need realistic data volumes or specific schema configurations, connect CodeBuild directly to a private Amazon RDS instance. Place the CodeBuild project inside a VPC, assign it private subnets and a security group, then add an inbound rule on the RDS security group allowing traffic from the CodeBuild security group.

# Add VPC configuration to an existing CodeBuild project
aws codebuild update-project \
  --name my-app-tests \
  --vpc-config vpcId=vpc-abc123,subnets=subnet-def456,securityGroupIds=sg-ghi789

Store database credentials in CI/CD secrets management and reference them in your buildspec using the env.secrets or env.parameter-store sections. Never hardcode connection strings in buildspec.yml.

Docker-based vs VPC-connected integration tests

Use this comparison to decide which approach fits your situation.

Start with Docker containers. Move to VPC-connected databases only when you have a specific reason, such as realistic data volume, driver-level behavior differences, or shared test fixtures that are impractical to containerize.

Speeding up test runs with caching and batch builds

Caching dependencies

Installing dependencies from scratch on every build is slow. A Python project with 50 packages can take two minutes on pip install alone. Use the CodeBuild cache to store installed packages in S3 and restore them at the start of the next build.

version: 0.2

cache:
  paths:
    - '/root/.cache/pip/**/*'
    - 'node_modules/**/*'

phases:
  install:
    commands:
      - pip install -r requirements.txt

The first build after enabling caching populates the cache. Subsequent builds skip re-downloading packages already stored. For a large Python project, this typically cuts install time from two minutes to under 20 seconds.

Parallel test runs with batch builds

If your test suite takes 15 minutes to run sequentially, a batch build can split it across multiple CodeBuild instances running simultaneously. Each task in a batch build has its own buildspec and runs independently. After tests pass, the next step is usually deployment. See Deploying with CodeBuild for how those stages connect.

version: 0.2

batch:
  fast-fail: true
  build-list:
    - identifier: unit_tests
      env:
        variables:
          TEST_SUITE: unit
      buildspec: buildspecs/unit-tests.yml

    - identifier: integration_tests
      env:
        variables:
          TEST_SUITE: integration
      buildspec: buildspecs/integration-tests.yml

    - identifier: security_scan
      buildspec: buildspecs/security-scan.yml

With fast-fail: true, a failure in any task cancels the others immediately. All three tasks appear in the CodeBuild console as separate build runs linked under the same batch build.

For environment-specific test behavior, such as running different test suites against dev vs staging, see Managing Environments in CI/CD. For pipeline-wide security design, see Secure CI/CD Pipelines.