Managing Secrets in CI/CD Pipelines

Why hardcoded secrets in pipelines are dangerous

A secret hardcoded in pipeline YAML is committed to version control. Every developer with repository access can read it. Every service that mirrors the repository (GitHub, Azure Repos, Bitbucket) stores it in their systems. If the repository is accidentally made public, the secret is public. If an attacker gains read access to the repository, they have the secret.

Even if the pipeline file is never committed with the literal secret (you use a variable reference like $(DB_PASSWORD)), the secret value must come from somewhere. Where it is stored, who can read it, and whether it is ever logged determines whether it is truly protected.

The threat model for pipeline secrets: secrets appear in pipeline logs if a script prints them, secrets stored in the CI/CD platform’s variable storage are exposed to admins of that platform, and long-lived credentials (service principal client secrets) remain valid until explicitly rotated.

The secret storage hierarchy

From most secure to least secure:

1. Azure Key Vault with OIDC or Managed Identity. Secrets stored in Key Vault, accessed by the pipeline using a short-lived token or managed identity. No long-lived credential needed. Full audit log in Key Vault. Secret rotation is independent of pipeline changes.
2. Azure DevOps Key Vault-backed variable groups. Variable group reads from Key Vault at runtime. The secret value is never stored in Azure DevOps. Masked in logs. Rotation in Key Vault takes effect immediately.
3. Azure DevOps secret pipeline variables / GitHub encrypted secrets. Encrypted at rest in the CI/CD platform. Masked in logs. Not shown in the UI after creation. Acceptable for values that do not need rotation management, but requires manual updates when rotating.
4. Hardcoded in pipeline YAML or committed to git. Never acceptable. Not even in a private repository.

Key Vault-backed variable groups in Azure DevOps

The recommended approach for most scenarios. Create a variable group in Azure DevOps that links to a Key Vault. The group lists which Key Vault secrets to expose. The pipeline accesses them by name as masked variables.

Setup steps: in Azure DevOps, go to Pipelines > Library > Variable groups > New variable group. Select “Link secrets from an Azure key vault as variables.” Choose the service connection and Key Vault, then select the specific secrets to include. Save the group.

In your pipeline YAML:

variables:
  - group: vg-myapp-prod-secrets  # Key Vault-backed group

stages:
  - stage: Deploy
    jobs:
      - deployment: DeployApp
        environment: env-prod
        strategy:
          runOnce:
            deploy:
              steps:
                - task: AzureWebApp@1
                  inputs:
                    azureSubscription: 'sc-myapp-prod'
                    appName: 'app-myapp-prod'
                    appSettings: >
                      -ConnectionStrings__DefaultConnection "$(DB_CONNECTION_STRING)"
                      -ApiKeys__ThirdParty "$(THIRDPARTY_API_KEY)"

GitHub Actions OIDC setup for Azure

With OIDC, GitHub Actions does not need a stored Azure client secret or certificate. GitHub’s identity provider issues a short-lived token at workflow runtime. Azure validates the token and grants access. The token expires in minutes.

Step 1: create a federated credential on the service principal:

# Create service principal (if it doesn't exist)
SP_ID=$(az ad sp create-for-rbac \
  --name sp-github-actions-prod \
  --role Contributor \
  --scopes /subscriptions/<sub-id>/resourceGroups/rg-prod \
  --query appId \
  --output tsv)

# Add federated credential for GitHub Actions
az ad app federated-credential create \
  --id $SP_ID \
  --parameters '{
    "name": "github-actions-prod",
    "issuer": "https://token.actions.githubusercontent.com",
    "subject": "repo:myorg/myrepo:environment:production",
    "audiences": ["api://AzureADTokenExchange"]
  }'

Step 2: store only three non-secret values in GitHub secrets — AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_SUBSCRIPTION_ID. No client secret stored anywhere.

Step 3: use the OIDC action in the workflow:

permissions:
  id-token: write
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4

      - name: Azure Login (OIDC)
        uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Deploy to App Service
        uses: azure/webapps-deploy@v3
        with:
          app-name: app-myapp-prod
          package: ./publish

The id-token: write permission is required for GitHub to request the OIDC token. Without it, the Azure Login action fails with a permission error.

Managed Identity for self-hosted pipeline agents

If you run self-hosted Azure Pipelines agents on Azure VMs or Azure Container Instances, assign a managed identity to the VM or container group. The pipeline agent can then authenticate to Key Vault, Azure Container Registry, and other Azure services without any stored credentials.

# Assign managed identity to the agent VM
az vm identity assign \
  --name vm-pipeline-agent-01 \
  --resource-group rg-pipeline-agents

# Get the managed identity's principal ID
PRINCIPAL_ID=$(az vm show \
  --name vm-pipeline-agent-01 \
  --resource-group rg-pipeline-agents \
  --query "identity.principalId" \
  --output tsv)

# Grant the identity access to Key Vault secrets
az keyvault set-policy \
  --name kv-myapp-prod \
  --object-id $PRINCIPAL_ID \
  --secret-permissions get list

In the pipeline, the Az CLI task automatically uses the managed identity when running on an agent with one assigned. No login step required — Azure authenticates the agent via the VM’s identity token, which is valid only on that machine.

Secret rotation without pipeline changes

One of the operational advantages of Key Vault-backed variable groups is decoupled rotation. The pipeline references a Key Vault secret by name. When the secret is rotated in Key Vault (new version created), the next pipeline run automatically gets the new value. No pipeline YAML change, no variable group update, no deployment needed to apply the rotation.

For Azure-managed credentials (storage account keys, service bus connection strings), enable automatic rotation using Key Vault’s built-in rotation functions that regenerate keys on a schedule. For third-party credentials, create a Key Vault secret rotation function or Azure Automation runbook that calls the third party’s API and stores the new credential in Key Vault.