SquadSquad

Azure Container Apps Deployment

⚠️ Experimental — Squad is alpha software. APIs, commands, and behavior may change between releases.

Azure Container Apps (ACA) is the lowest-friction path to running Squad in production on Azure. ACA provides managed KEDA scaling, managed TLS/ingress, and native workload identity — so you get autoscaling and secretless authentication without managing a Kubernetes control plane.

This guide takes you from zero to a running Squad agent that polls a GitHub repository and executes issues automatically.

Prerequisites:

  • An Azure subscription with Contributor access
  • az CLI 2.53+ and the containerapp extension: az extension add --name containerapp
  • A container image pushed to ACR or GHCR (see Container Image reference)
  • A GitHub repository with Squad initialized (squad init)

Architecture Overview

GitHub Issues


┌─────────────────────────────────────────┐
│  Azure Container Apps Environment       │
│                                         │
│  ┌────────────────────────────────────┐ │
│  │  Squad Agent Container App         │ │
│  │  • KEDA HTTP or queue scaler       │ │
│  │  • Workload identity (no PATs)     │ │
│  │  • Key Vault secret references     │ │
│  └────────────────────────────────────┘ │
│                                         │
│  ┌────────────────────────────────────┐ │
│  │  ACA Jobs (optional)               │ │
│  │  • One-shot issue dispatch         │ │
│  └────────────────────────────────────┘ │
└─────────────────────────────────────────┘


  Azure Key Vault   Azure Monitor / Log Analytics

Step 1 — Resource Setup

# Variables — adjust to your environment
RESOURCE_GROUP="rg-squad-prod"
LOCATION="eastus2"
ACA_ENV="squad-aca-env"
LOG_WORKSPACE="squad-logs"
ACR_NAME="squadregistry"        # must be globally unique
KEYVAULT_NAME="squad-kv-prod"   # must be globally unique
APP_NAME="squad-agent"
IDENTITY_NAME="squad-agent-id"

# Resource group
az group create --name $RESOURCE_GROUP --location $LOCATION

# Log Analytics workspace (required for ACA environment)
az monitor log-analytics workspace create \
  --resource-group $RESOURCE_GROUP \
  --workspace-name $LOG_WORKSPACE

LOG_WORKSPACE_ID=$(az monitor log-analytics workspace show \
  --resource-group $RESOURCE_GROUP \
  --workspace-name $LOG_WORKSPACE \
  --query customerId -o tsv)

LOG_WORKSPACE_KEY=$(az monitor log-analytics workspace get-shared-keys \
  --resource-group $RESOURCE_GROUP \
  --workspace-name $LOG_WORKSPACE \
  --query primarySharedKey -o tsv)

# Container Apps environment
az containerapp env create \
  --name $ACA_ENV \
  --resource-group $RESOURCE_GROUP \
  --location $LOCATION \
  --logs-workspace-id $LOG_WORKSPACE_ID \
  --logs-workspace-key $LOG_WORKSPACE_KEY

Reference: Azure Container Apps environments


Step 2 — Managed Identity

Using a user-assigned managed identity is the recommended authentication strategy. It replaces long-lived GitHub PATs for Azure service access and enables OIDC federation for GitHub API calls.

# Create user-assigned managed identity
az identity create \
  --name $IDENTITY_NAME \
  --resource-group $RESOURCE_GROUP

IDENTITY_CLIENT_ID=$(az identity show \
  --name $IDENTITY_NAME \
  --resource-group $RESOURCE_GROUP \
  --query clientId -o tsv)

IDENTITY_RESOURCE_ID=$(az identity show \
  --name $IDENTITY_NAME \
  --resource-group $RESOURCE_GROUP \
  --query id -o tsv)

Reference: Managed identities in Azure Container Apps

ACR pull with managed identity

Grant the identity AcrPull on your registry so the container can pull the Squad image without a stored password:

ACR_ID=$(az acr show --name $ACR_NAME --resource-group $RESOURCE_GROUP --query id -o tsv)

az role assignment create \
  --assignee $IDENTITY_CLIENT_ID \
  --role AcrPull \
  --scope $ACR_ID

Step 3 — Key Vault and GitHub Token Secret

Store the GitHub token (or any other secrets) in Key Vault rather than in the container environment.

# Create Key Vault
az keyvault create \
  --name $KEYVAULT_NAME \
  --resource-group $RESOURCE_GROUP \
  --location $LOCATION \
  --enable-rbac-authorization true

# Grant the managed identity read access to secrets
KV_ID=$(az keyvault show --name $KEYVAULT_NAME --resource-group $RESOURCE_GROUP --query id -o tsv)

az role assignment create \
  --assignee $IDENTITY_CLIENT_ID \
  --role "Key Vault Secrets User" \
  --scope $KV_ID

# Store the GitHub token
az keyvault secret set \
  --vault-name $KEYVAULT_NAME \
  --name github-token \
  --value "<your-github-token>"

Reference: Key Vault secret references in Azure Container Apps


Step 4 — Deploy the Container App

# Use a versionless URI so ACA automatically picks up rotated secrets.
# az keyvault secret show returns a versioned URI — strip the version segment.
KV_SECRET_URI="https://${KEYVAULT_NAME}.vault.azure.net/secrets/github-token"

az containerapp create \
  --name $APP_NAME \
  --resource-group $RESOURCE_GROUP \
  --environment $ACA_ENV \
  --image "${ACR_NAME}.azurecr.io/squad-agent:latest" \
  --registry-server "${ACR_NAME}.azurecr.io" \
  --registry-identity $IDENTITY_RESOURCE_ID \
  --user-assigned $IDENTITY_RESOURCE_ID \
  --secrets "github-token=keyvaultref:${KV_SECRET_URI},identityref:${IDENTITY_RESOURCE_ID}" \
  --env-vars \
    "GITHUB_TOKEN=secretref:github-token" \
    "SQUAD_DEPLOYMENT_MODE=agent-per-node" \
    "OTEL_SERVICE_NAME=squad-agent" \
  --cpu 0.5 \
  --memory 1.0Gi \
  --min-replicas 0 \
  --max-replicas 5

Versionless URI and rotation: Using the versionless secret URI (…/secrets/<name> with no version segment) means ACA will resolve the latest active version of the secret on each new revision. If you rotate your GitHub token in Key Vault, create a new container app revision to pick up the new value. Using a versioned URI pins the secret and will not pick up rotations automatically.

No ingress required. Squad agents poll GitHub; they do not need to receive inbound HTTP traffic. Omit --ingress (or use --ingress internal) to restrict access to the ACA environment.


Step 5 — KEDA Scaling

Azure Container Apps has built-in KEDA support. You can add scaling rules after creation or inline during az containerapp create.

Option A — ACA built-in HTTP scaler

For HTTP-triggered workflows (webhook dispatch, future --remote mode — see #1189):

az containerapp update \
  --name $APP_NAME \
  --resource-group $RESOURCE_GROUP \
  --scale-rule-name http-scaler \
  --scale-rule-type http \
  --scale-rule-http-concurrency 10

Option B — KEDA external scaler (GitHub issue queue)

Use the Squad KEDA external scaler to scale based on GitHub issue queue depth. This matches the pattern documented in KEDA Autoscaling.

The KEDA external scaler must be deployed as a separate container in the same ACA environment:

# Deploy external scaler service
az containerapp create \
  --name squad-external-scaler \
  --resource-group $RESOURCE_GROUP \
  --environment $ACA_ENV \
  --image "ghcr.io/bradygaster/squad-external-scaler:latest" \
  --env-vars "PORT=8080" \
  --ingress internal \
  --target-port 8080 \
  --min-replicas 1 \
  --max-replicas 1

Then add a custom KEDA scaling rule to the Squad agent app using the KEDA external scaler protocol:

az containerapp update \
  --name $APP_NAME \
  --resource-group $RESOURCE_GROUP \
  --scale-rule-name github-queue \
  --scale-rule-type external \
  --scale-rule-metadata \
    "scalerAddress=squad-external-scaler:8080" \
    "owner=<your-org>" \
    "repo=<your-repo>" \
    "labels=squad:ready" \
    "targetQueueLength=5" \
  --scale-rule-auth "trigger=github-token" "secretRef=github-token"

Reference: KEDA scaling rules in Azure Container Apps

KEDA scaler PAT rotation: The KEDA external scaler uses github-token (a GitHub PAT) to poll the issue queue — this is a separate credential from the Squad agent’s own GITHUB_TOKEN (which can also be a PAT). Both tokens need to be rotated independently. When you rotate the scaler’s PAT in Key Vault and update the ACA secret, redeploy the scaler container to pick up the new value. There is no automatic token refresh in the current scaler implementation. See KEDA Autoscaling for scaler configuration details.

Scaling to zero — cold start tradeoffs

Setting --min-replicas 0 eliminates idle cost but introduces a cold start penalty (typically 15–45 seconds for a Node.js container). During a cold start, issues that arrive are queued by KEDA and picked up when the first replica becomes ready.

For time-sensitive workloads, set --min-replicas 1 to keep one warm replica.


ACA Jobs — One-Shot Issue Dispatch

For triggered, one-shot agent runs (e.g., a single issue dispatched by a webhook), use ACA Jobs instead of a long-running Container App:

az containerapp job create \
  --name squad-job \
  --resource-group $RESOURCE_GROUP \
  --environment $ACA_ENV \
  --image "${ACR_NAME}.azurecr.io/squad-agent:latest" \
  --registry-server "${ACR_NAME}.azurecr.io" \
  --registry-identity $IDENTITY_RESOURCE_ID \
  --user-assigned $IDENTITY_RESOURCE_ID \
  --trigger-type Manual \
  --replica-timeout 1800 \
  --replica-retry-limit 1 \
  --replica-completion-count 1 \
  --parallelism 1 \
  --secrets "github-token=keyvaultref:${KV_SECRET_URI},identityref:${IDENTITY_RESOURCE_ID}" \
  --env-vars \
    "GITHUB_TOKEN=secretref:github-token" \
    "SQUAD_DEPLOYMENT_MODE=agent-per-node"

Trigger a job run manually:

az containerapp job start \
  --name squad-job \
  --resource-group $RESOURCE_GROUP

Future: Remote dispatch (squad --remote) is being designed in #1189. Once available, ACA Jobs will be the recommended execution backend for remote dispatch.


State Persistence Caveats

By default, Squad uses the local state backend — files written to .squad/ inside the container. Container filesystem is ephemeral: state is lost when a replica restarts.

For durable state:

  1. Orphan/two-layer backend (recommended): State is committed to an orphan Git branch; persists across pod restarts. Requires GITHUB_TOKEN write scope.
  2. Azure Files volume mount: Mount an Azure Files share to /app/.squad/ for shared file-system persistence across replicas.
  3. Single replica: If scaling to one replica (min=max=1), ephemeral state survives in-memory but not across restarts.

⚠️ Multi-replica safety: The local state backend is not safe for concurrent writes from multiple replicas. Use the orphan or two-layer backend, or constrain to one replica. See State Backends and #1402.


Troubleshooting

Missing or invalid GITHUB_TOKEN

Symptom: Squad starts but immediately logs 401 Unauthorized or GitHub API rate limit exceeded for an invalid token.

# Check live logs
az containerapp logs show \
  --name $APP_NAME \
  --resource-group $RESOURCE_GROUP \
  --type console \
  --follow

# Verify the secret is set
az containerapp secret list \
  --name $APP_NAME \
  --resource-group $RESOURCE_GROUP

Ensure the Key Vault secret URI is correct and the managed identity has Key Vault Secrets User on the vault. Check the identity assignment:

az containerapp identity show \
  --name $APP_NAME \
  --resource-group $RESOURCE_GROUP

KEDA scaler not triggering

Symptom: Issues are labeled squad:ready but no new replicas start.

  • Verify the external scaler container is running: az containerapp show --name squad-external-scaler --resource-group $RESOURCE_GROUP --query properties.runningStatus
  • Check scaler logs for GitHub API errors (rate limit, auth failure).
  • Confirm the targetQueueLength is below the current issue count.
  • Check the ACA scaling logs in Log Analytics: az monitor log-analytics query --workspace $LOG_WORKSPACE_ID --analytics-query "ContainerAppConsoleLogs_CL | where ContainerName_s == 'squad-agent' | take 50"

Cold start delays

Symptom: First issue pickup after scale-from-zero is slow (15–45 s typical).

  • This is expected behavior — Squad has no readiness endpoint yet (see #1577). ACA restarts exited containers automatically; no HTTP probe is needed.
  • Set --min-replicas 1 to keep one warm replica if latency matters.
  • Use node:22-alpine base image (not node:22) for a smaller, faster startup.

Container exits immediately

Symptom: Container starts and exits with code 1 within seconds.

  • Check that GITHUB_TOKEN is injected correctly (test with az containerapp exec if available).
  • Verify the image tag matches what was pushed to ACR.
  • Check startup logs: az containerapp logs show --name $APP_NAME --resource-group $RESOURCE_GROUP --type system

Current Limitations and Future Work

FeatureStatus
--remote dispatch (webhook-triggered issue execution)RFC — #1189
External state gap for multi-replica deploymentsDesign — #1402
FSStorageProvider rootDir bug (no env-var override; set rootDir in config.json)Open — #1555
HTTP health/readiness endpointsPlanned — #1577
ACA Dynamic Sessions / Sandbox executionPlanned — #1564

Further Reading