Multi-region (V2) is on main and opt-in: active-standby AWS across eu-central-1 (primary) + us-east-1 (standby), via ./scripts/bootstrap-multiregion.sh. Single-region setups are unaffected. See docs/multi-region.md.

Agentic Development Platform (ADP) is on main and opt-in: extends the AI/ML stack into a first-class agent layer for both dev workflow (scaffold/code/test/review) and ops (cost/incidents/security), with a human-in-the-loop approval gate for any mutating action. Enable with ./scripts/bootstrap.sh --adp on AWS, or ./scripts/bootstrap-ai.sh --adp locally. See docs/agentic-platform.md.

Compatibility

Component	Tested version
Backstage	v1.50.4
Kubernetes	1.32 (EKS) · 1.33.1 (Kind)
Helm	3.x / 4.x
Kind	≥ 0.27
ArgoCD	v3.4 (chart 9.5.13)
Terraform	≥ 1.5 (CI pins 1.10.5)
Go (hello-service)	1.26
Node.js (Backstage)	22 or 24 (CI builds on 24; the MCP servers run on Node 20)

---

What You Get

Capability	Details
Developer portal	Backstage v1.50.4 — catalog, TechDocs, Tech Radar (92 entries), custom scaffolder actions
Software templates	64 templates: 12 blessed golden-path (Node.js, Python, Go, Ruby, JVM, React, LLM App, LangGraph Agent, Team namespace, Create namespace, Add-secret, Decommission) + 52 advanced (infra, QA, mobile, AI/ML, multi-region, observability). Adding one is a single line in backstage/catalog/all-templates.yaml (63 there; deploy-to-kind is local-only, registered in app-config.local.yaml)
QA / test templates	18 testing scaffold types — Playwright, k6, Pact, Newman, ZAP, Datadog, Visual Regression, Accessibility, Cucumber, Appium, Chaos Mesh, Stryker Mutation, Testcontainers, DeepEval, Unit, Component, IaC, Flutter Integration. See CLI Reference
Team isolation	Per-team namespace (quota + LimitRange + NetworkPolicy + ArgoCD AppProject), per-team SecretStore + Grafana folder, Kyverno-injected idp:team tags. See docs/team-management.md
Mobile platform	7 mobile golden-path templates (Android/iOS/Flutter/SDK/Code Signing/App Store/Device Farm) + a 5-check mobile scorecard whose tiers gate on named requirements rather than a count. See docs/mobile-platform.md
Golden-path chart	One reusable Helm chart for all services — health checks, metrics, RBAC, PodDisruptionBudget, optional Argo Rollouts canary
Shift-left quality	Bronze/Silver/Gold scorecard (19 checks — 14 for non-AI entities, plus 5 AI-governance checks including a LiteLLM virtual key and spend budget) in Tech Insights + Grafana, with one predicate package shared by the portal and the backend; a compliance watcher posts to Slack and files a Jira ticket when a check regresses; PR gates for coverage/vuln/static analysis; ArgoCD PreSync contract gate. See docs/shift-left-leadership.md
Engineering Intelligence	An /engineering-intelligence dashboard scoring Platform, Developer Experience, Quality, Reliability, AI Engineering, Security and FinOps health, and placing the organisation on a five-level maturity model, from the telemetry the platform already produces — every score decomposing into evidence that names its metric, source and timestamp. Dimensions with no data source say so and return no number rather than a plausible one. Keeps its own snapshot history, since Prometheus retains 24h locally (and 30d raw in Thanos on AWS). See docs/engineering-intelligence/
AI/ML platform	KAgent agents (Claude + GPT-4o) + MLflow + 8 MCP servers (IDP, QA, Contract, GitHub, Cost, ArgoCD, Incident, Security) + Model Serving API + AI scorecard + RAG search over TechDocs. All MCP tool traffic and model calls route through a single AI Gateway (agentgateway, standalone — ~9 MiB) rather than direct per-server wiring; local admin console at ai-gateway.idp.local. Model calls then go through LiteLLM, which holds the Anthropic key and Bedrock's IRSA credentials and adds virtual keys and spend tracking, surfaced as the LiteLLM Spend page (on by default on AWS, bootstrap-ai.sh --litellm locally). In-portal KAgent and MLflow pages (agents/MCP servers; experiments, runs and the model registry). See docs/ai-assistant.md, ADR-0007 and ADR-0008
LLM observability	Langfuse — prompt/completion, token counts, cost and latency per agent run, plus versioned agent prompts and a CI drift gate. KAgent exports OTLP directly and all 8 MCP servers trace their tool calls; surfaced as the AI Observability page in Backstage. Self-service for your own services via the enable-langfuse-tracing and llm-app-langfuse templates, with a per-service Langfuse entity tab. Installed by default on both targets by bootstrap-ai.sh (part of --with-ai on AWS); --skip-langfuse opts out. See docs/ai-assistant.md
Observability	Prometheus + Grafana (local) / CloudWatch + Grafana (AWS); Loki + Tempo; PagerDuty; Sloth SLOs; DORA entity tab per-team; FinOps cost overview. See docs/dora-finops.md
Datadog	Cluster-wide Agent (infra metrics, logs, APM intake, AWS only) alongside Prometheus/Grafana; dd-trace on the Backstage backend; Datadog entity tab (dashboard/monitor/SLO status); enable-datadog-apm scaffolder template. See docs/sre-reliability.md
Infrastructure	Terraform for foundation (EKS, VPC, ECR, IAM/OIDC, RDS, S3) + Crossplane for per-service resources (S3, RDS, MSK, DynamoDB, SQS) via ArgoCD-reconciled Claims. See docs/crossplane-vs-terraform.md
Multi-region V2	Active-standby eu-central-1 + us-east-1, opt-in. See docs/multi-region.md
Agentic Development Platform (ADP)	Opt-in agent layer on top of the AI/ML platform — dev-workflow agents (scaffold/code/test/review) and ops agents (cost/incidents/security), gated by a human-in-the-loop approval layer. bootstrap.sh --adp (AWS) or bootstrap-ai.sh --adp (local). See docs/agentic-platform.md
CI/CD	GitHub Actions — test → Docker build → ECR push → Helm deploy to EKS

Quick Start

Prerequisites

Path	Install first
Local (Kind)	git, docker, kind ≥ 0.27, kubectl, helm ≥ 3.14 — brew install kind kubectl helm docker on macOS
AWS	Everything above, plus aws CLI (run aws configure), terraform ≥ 1.5, jq

Local machine sizing: measured on an 8 CPU / 13 GB VM (2026-08-22). The core
platform is ~6.9 GB; Langfuse adds ~2.2 GB, KAgent plus one agent
~0.5 GB (each extra agent ~200 MB), MLflow ~0.4 GB. Everything at once is
~11.8 GB, so give Docker/Rancher Desktop 16 GB (24 GB physical) for the
full stack, or 13 GB (16 GB physical) for the core platform plus one of
Langfuse / KAgent / MLflow — which is the realistic ceiling on a 16 GB machine.
Both Kind nodes share one VM. Below ~200 MB available the API server stops
answering, so stop adding components under ~1.5 GB spare. Failure thresholds,
per-layer costs and how to trim: Machine requirements.

go and Node.js are only needed if you want to build the idp CLI / run Backstage outside Docker — setup.sh builds the CLI for you automatically if Go is present, and skips it with a warning otherwise. Full checklists: Local Setup · AWS Deployment Guide.

# 1. Click "Use this template" on GitHub, then clone your new repo
git clone https://github.com/moatazeldebsy/backstage-platform-template.git && cd backstage-platform-template

# 2. Run the one script you need — it does everything else for you
./scripts/setup.sh

setup.sh is the only command you run by hand on a fresh clone. It's a one-time personalization + dispatcher: it replaces placeholders across the repo with your GitHub org/cluster name, creates .env files, then asks local or AWS and triggers the real installer for you automatically — bootstrapping has to happen after personalization, otherwise ArgoCD and the catalog would still point at unresolved placeholders.

#	Runs	Automatic?
1	setup.sh — personalises placeholders, asks local or AWS	You run this
2a (local)	bootstrap-local.sh — the actual Kind cluster + platform installer (~15–20 min)	Auto, by setup.sh
2b (local)	bootstrap-local.sh --start-backstage — builds + starts Backstage (~2 min)	Auto, if you answer Y to "Start Backstage now?"
2 (AWS)	bootstrap.sh — Terraform → EKS → core platform (~40–70 min). AI/ML is opt-in: add --with-ai, or --adp for the agentic layer too	Auto, by setup.sh
3 (optional, both targets)	bootstrap-ai.sh — adds KAgent + MLflow + Langfuse + MCP servers (--aws on EKS)	Manual on local and AWS alike

Don't run setup.sh and then bootstrap-local.sh. Step 2a above is automatic — setup.sh has already run it by the time it finishes. Running it again just repeats a 15–20 minute install. If setup.sh printed the "Local IDP platform is up" banner with the access URLs, your cluster is up and the next (optional) step is bootstrap-ai.sh.

bootstrap-local.sh (and bootstrap.sh/bootstrap-multiregion.sh on AWS) is also the script you run standalone for every day-2 operation afterwards — recreating the cluster, --destroy, --start-backstage, --print-urls, etc. You don't re-run setup.sh for those; see Scripts Reference for the full breakdown of what each script owns.

For AWS, first copy terraform/terraform.tfvars.example to terraform/terraform.tfvars and set github_org, aws_region, cluster_name, then run ./scripts/verify-secrets.sh to confirm your credentials/secrets are in place before bootstrap.sh.

After local bootstrap, Backstage is at http://backstage.idp.local and hello-service at http://hello-service.idp.local. Day-2 commands (re-running any step standalone), full walkthroughs, and the AI/ML step: Scripts Reference · Local Setup · AWS Deployment Guide.

Local access URLs

Written automatically to /etc/hosts by bootstrap-local.sh (you may need sudo on first run):

Service	URL	Default credentials
Backstage	http://backstage.idp.local (or http://localhost:3000)	— (guest mode)
hello-service	http://hello-service.idp.local	—
Grafana	http://grafana.idp.local	admin / admin
ArgoCD	http://argocd.idp.local	admin / (run kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d)
Prometheus	http://prometheus.idp.local	—
AlertManager	http://alertmanager.idp.local	—
Pushgateway	http://pushgateway.idp.local	—
Argo Workflows	http://argo-workflows.idp.local	—
OpenCost	http://opencost.idp.local	—
AI Assistant / AI Search	http://backstage.idp.local/ai-assistant · /ai-search	requires bootstrap-ai.sh (+ VOYAGE_API_KEY for search)
KAgent UI / MLflow UI	http://kagent.idp.local · http://mlflow.idp.local (also surfaced in Backstage at /kagent · /mlflow)	requires bootstrap-ai.sh
AI Gateway (admin console)	http://ai-gateway.idp.local — routes, MCP targets, model list, Chat/Tool Playgrounds	requires bootstrap-ai.sh; local only, no AWS Ingress for this port (--skip-gateway opts out)
LiteLLM (spend dashboard)	http://litellm.idp.local (also surfaced in Backstage as the LiteLLM Spend page)	requires bootstrap-ai.sh --litellm locally; the nav item stays hidden until LiteLLM is installed
AI Observability / Langfuse UI	http://backstage.idp.local/langfuse · http://langfuse.idp.local	installed by default by bootstrap-ai.sh (--skip-langfuse opts out); Langfuse admin password in the langfuse-init Secret
MCP Servers (8)	http://<name>-mcp-server.idp.local/healthz — idp, qa, contract, github, cost, argocd, incident, security	requires bootstrap-ai.sh
Approval Service / Agent Event Router	http://approval-service.idp.local · http://agent-event-router.idp.local	requires bootstrap-ai.sh --adp
Traces (Tempo) / Logs (Loki) / Argo Rollouts	Traces and logs via Grafana Explore (Tempo has no UI; tempo.idp.local/v1/traces is a POST-only OTLP endpoint) · http://argo-rollouts.idp.local	auto-deployed by bootstrap-local.sh, but Loki and Tempo are scaled to 0 locally by default — scale up to use them (Local Setup)
Local registry	localhost:5003	— (no auth)

Third-party integrations — bring your own accounts

You do not need any third-party account to run this platform. Every
integration below is optional and fails soft: the portal boots, the cluster comes
up, and the relevant tab renders an empty state rather than an error. Nothing is
stubbed or mocked — the config, proxy wiring and secret plumbing are real, so if
you do have an account, filling in one variable is all that's needed.

Credentials are supplied by local/backstage/.env locally (start from
local/backstage/.env.example, which documents each one) and by AWS Secrets
Manager → External Secrets on EKS.

Integration	What you need	Without it
GitHub (catalog, scaffolder)	PAT in GITHUB_TOKEN, scopes repo, read:org, workflow, delete_repo	Catalog import and scaffolding to real repos don't work — the rest of the portal is unaffected. In practice this is the one worth setting.
GitHub OAuth	OAuth App → AUTH_GITHUB_CLIENT_ID / AUTH_GITHUB_CLIENT_SECRET	Guest mode only — no "Sign in with GitHub"
SonarCloud / Snyk	Free-tier tokens → SONAR_TOKEN / SNYK_TOKEN	Security tab renders empty; scaffolded CI skips those steps and stays green
Datadog	DD_API_KEY + DD_APP_KEY	Datadog tab renders empty. On AWS these also drive the Datadog Agent and APM
PagerDuty	Read-only REST API key → PAGERDUTY_TOKEN	On-call tab renders empty.
Jira	JIRA_URL + JIRA_TOKEN = Base64(email:api_token)	Issues tab renders empty.
Voyage AI	VOYAGE_API_KEY (free tier: 200M tokens/month)	/ai-search returns HTTP 503. Everything else in the AI layer still works
Firebase Test Lab / GCP	Service-account JSON, base64 → GCP_SERVICE_ACCOUNT_KEY	The mobile device-farm and Flutter test-suite templates scaffold fine but their CI can't authenticate
LambdaTest	Username + access key → LT_USERNAME / LT_ACCESS_KEY	The device-farm, Appium and Playwright templates scaffold fine, but their LambdaTest jobs can't authenticate
BrowserStack / Sauce Labs	Username + access key → BROWSERSTACK_* / SAUCE_*	Those device-farm providers scaffold fine but their CI can't authenticate
Grafana, ArgoCD	—	Auto-populated by bootstrap-local.sh; no account needed

The screenshots throughout these docs were taken on an instance with several of
these configured, so some tabs show live data that will be empty on a fresh
install until you add your own credentials.

Platform Summary

Layer	Local	AWS
Compute	Kind (Kubernetes in Docker)	Amazon EKS 1.32
Container registry	Local registry (localhost:5003)	Amazon ECR
Ingress	nginx ingress controller	AWS Load Balancer Controller (ALB)
CI / CD	GitHub Actions → idp:deploy-local Backstage action	GitHub Actions (OIDC → ECR → EKS)
IaC (foundation)	—	Terraform (EKS, VPC, ECR, IAM, RDS, S3, Secrets Manager)
IaC (per-service)	—	Crossplane (S3, RDS, MSK, DynamoDB, SQS) — Claims in Git, reconciled by ArgoCD
Deployment	Helm (helm/service-template)	Helm (helm/service-template)
Developer portal	Backstage (Docker Compose)	Backstage (EKS)
Observability	Prometheus + Grafana	CloudWatch + Grafana + Datadog Agent (infra/APM)
LLM observability	Langfuse (default) — in-cluster Postgres + ClickHouse + MinIO	Langfuse (default) — RDS + S3 via Terraform, IRSA-scoped

Platform Planes

The planes diagram at the top of this page is the systems
view: five planes, each independently installable. The core IDP runs without the AI and
Observability planes, and the AI plane holds no privilege the planes above it do not already
grant.

AWS Architecture

Full layer-by-layer breakdown: docs/architecture.md.

AWS Architecture — Multi-Region (V2, opt-in)

Active-standby across eu-central-1 (primary) and us-east-1 (warm standby), deployed with
./scripts/bootstrap-multiregion.sh. Single-region setups are unaffected.

Topology, DR tiers, and the six rollout phases: docs/multi-region.md.

What it costs on AWS

Measured against a running idp-mvp cluster in us-east-1 on 2026-08-14, at
on-demand list prices, for continuous 24/7 running — that is, before the
overnight scale-down enable_cost_optimizer performs by default.
Local is free — this is only the EKS path. These are AWS infrastructure
charges only; see what the table excludes below
before treating the total as your bill.

This is a measurement, not the repo default. The table reflects a running
idp-mvp cluster with the node group scaled up to six t3.large.
terraform/variables.tf ships node_group_desired_size = 1 (min 0, max 2), so
a bootstrap using the defaults costs substantially less — and will not fit the
full stack. Scale the node group deliberately rather than inferring it from
this table.

Component	Qty	~$/month
EKS control plane	1	73
Worker nodes (t3.large)	6	364
NAT gateway	1	33 + data
Application Load Balancers — core	4	66
RDS for Backstage (db.t3.micro)	1	12
S3 / ECR / Secrets Manager / CloudWatch	—	~15
Core platform subtotal		~565
Application Load Balancers — AI/ML	11	181
RDS for Langfuse (db.t4g.micro)	1	11
S3 for MLflow + Langfuse artifacts	2	~2
AI/ML layer subtotal (--with-ai)		~195
Total with AI/ML		~760

The load balancers are the surprise. Every ALB is ~$16/month before traffic,
and the AI/ML layer creates eleven of the fifteen — one per MCP server, plus
KAgent, the IDP assistant, MLflow and Langfuse. That is more than the RDS
instances and S3 combined, and it is why the AI layer is opt-in
(./scripts/bootstrap.sh --with-ai) rather than default.

Ways to spend less, roughly in order of effect:

• Skip the AI/ML layer. Saves ~$195/month. enable_ai and enable_langfuse
  gate the infrastructure too, so nothing is provisioned for it.
• Leave enable_cost_optimizer = true (the default). Scales nodes to zero
  and stops RDS overnight — roughly halves the node and RDS lines if you only
  work office hours. Because it is on by default, a stock deployment on an
  office-hours schedule already lands well under the table above, which measures
  the optimizer-disabled case.
• Drop the node count. Six t3.large is sized for the full stack including
  AI; the core platform alone fits in fewer. Note the constraint documented on
  node_instance_types: nodes are sized by pod IP capacity, not CPU/RAM.
• Tear down when idle. ./scripts/cleanup.sh removes everything including
  the orphaned ALBs that a bare terraform destroy leaves behind.

What the table excludes

The total is the platform's own AWS footprint. It is not an all-in run rate:

Not counted	Why it can matter
Data transfer and NAT data processing	Charged per GB on top of the NAT hourly rate. Image pulls and cross-AZ traffic dominate it, so it scales with your workload, not with the platform
LLM API spend	KAgent runs Claude and GPT-4o, and AI Search needs a VOYAGE_API_KEY. Those are Anthropic / OpenAI / Voyage bills, not AWS — and on an agent-heavy platform they can exceed the infrastructure. This is precisely what the Langfuse page exists to show you
Datadog	Third-party SaaS priced per host and per ingested GB, alongside the Prometheus/Grafana stack that is included
Multi-region V2	A standby region is close to a second copy of the infrastructure. See docs/multi-region.md
Your own services	Everything above is the platform. Whatever your teams scaffold onto it is additional
Savings Plans / Reserved Instances	List prices only. Committed-use discounts take a meaningful cut off the node line

Verify against the AWS pricing calculator for your
region before committing — this table is a measurement of one cluster, not a
quote.

How It Works — Interaction Flows

Channel	Who	Entry point
1 — CLI	Developer	idp scaffold service / idp ai "list templates" → Scaffolder Engine → GitHub repo
2 — Backstage Portal	Developer / Platform Engineer	Software Catalog, 64 templates, TechDocs, Tech Radar, AI Assistant, DORA tab, Tech Insights scorecard
3 — AI Agent / MCP	AI Agent (KAgent + Claude / GPT-4o)	IDP MCP Server, QA MCP Server, Contract MCP Server → Platform APIs

Screenshots

All shots are from live clusters — mostly a local Kind cluster brought up with ./scripts/setup.sh + ./scripts/bootstrap-ai.sh, plus a few from the AWS EKS path. No mock-ups.

The portal

The Platform Dashboard is the landing page: catalog counts, platform-wide DORA, and every service at a glance.

Software Catalog	Teams
Every service, API, MCP server and test suite, owned and tagged	8 teams, each with its own namespace, SecretStore and Grafana folder

Each entity page carries the platform's own tabs — TechDocs, Kubernetes, DORA, Scorecard, Security, Datadog, Trivy, SLOs:

TechDocs on the entity	Scorecard on the entity

Golden path — scaffold → repo → deploy

64 templates in the Scaffolder, filtered by category, tag or owner:

Scaffolder task running	The repo it produced
Generate → push to GitHub → register in catalog → run the first job	CI workflow, Dockerfile, catalog-info.yaml, TechDocs — all wired

Templates are wizards, not a wall of fields — and the last step opens the GitOps PR that puts the new service under ArgoCD:

Template wizard (LLM App)	The GitOps PR it opened
Model, effort level, trace sampling — chosen up front, wired into the skeleton	An ApplicationSet auto-discovers the service into services-dev on merge

Self-service infrastructure is the same flow — a Crossplane Claim committed to Git instead of a Terraform PR:

ArgoCD app-of-apps	Argo Rollouts canary

Shift-left quality

Bronze / Silver / Gold tiers across every service, with the cheapest unfilled check called out as the next action:

SLOs and error budgets	QA platform metrics
Sloth multi-window burn-rate, live from Prometheus	E2E pass rate, k6 p95 latency and error rate per run

Those SLOs aren't hand-written YAML — a template generates the Sloth definitions and burn-rate alerts and opens the PR:

AI/ML platform and agents

The AI Assistant answers in plans, not prose — it maps your intent onto the actual templates on the platform and asks for exactly the inputs they need:

Ask it anything	It plans the scaffold

…and then it actually runs the scaffolder — repo, deploy target, task ID and the suggested next steps come back in the same chat:

KAgent agents	MCP servers and model configs

The MLflow page gives experiment tracking and the model registry the same in-portal treatment — experiments, recent runs and registered models, read live from the MLflow API, without leaving the catalog:

Agent Approvals (HiTL gate)	MLflow's own UI
Every mutating agent action waits for a human — or an auto-approve policy	One click away at mlflow.idp.local, for the deep-dive views

Semantic search over templates, components and TechDocs (Voyage AI + pgvector):

Prometheus tells you that an agent ran. Langfuse tells you what it cost — prompt and completion, tokens, latency and spend per run, without leaving the portal:

AI Observability in Backstage	The full Langfuse UI
Cost and token usage per model, and recent agent runs with latency	Cost by model and environment, per-user spend, and trace drill-down

Observability, DORA and FinOps

DORA metrics	FinOps cost overview
Four keys platform-wide and per service, with performance bands	OpenCost spend by namespace, team or container

Cost Calculator	Grafana — IDP services
Estimate a service's monthly cost before scaffolding it	Request rate, CPU/memory and restarts, filtered by catalog entity

Incidents are records, not Slack threads — auto-filed from Alertmanager, severity-filtered, and feeding MTTR back into DORA:

More screens — Tech Radar, onboarding, Learning Center, API explorer, Copilot metrics, admin, activity feed, search, support

Tech Radar (92 entries)	API Explorer

OpenAPI spec on the entity	Onboarding

Learning Center	Copilot metrics

Admin	Activity feed

Search	Support

Project Structure

backstage-platform-template/
├── scripts/                    # setup.sh · bootstrap-local.sh · bootstrap-ai.sh · cleanup.sh
├── backstage/
│   ├── app/                    # Backstage monorepo (v1.50.4)
│   ├── catalog/templates/      # 64 golden-path templates
│   ├── app-config.yaml         # base config
│   ├── app-config.local.yaml   # Kind overrides
│   └── app-config.aws.yaml     # EKS overrides
├── helm/service-template/      # single reusable Helm chart
├── services/hello-service/     # reference Go service
├── kubernetes/                 # namespaces · RBAC · ArgoCD app-of-apps · KAgent CRDs
├── local/                      # Kind config · nginx values · Docker Compose
├── aws/                        # EKS-specific: ArgoCD values · External Secrets · Crossplane
├── terraform/                  # EKS · VPC · ECR · IAM · IRSA
├── cli/                        # `idp` CLI (Go)
└── docs/                       # Architecture · golden path · runbooks

---

idp CLI

Built automatically by setup.sh (make cli-build → ./bin/idp). Scaffolds services and 18 types of test suites via the Backstage API when reachable, or locally otherwise:

idp scaffold service --name my-svc --type nodejs           # nodejs | python | go
idp scaffold test-suite --name my-e2e --type playwright --service my-svc
idp doctor                                                  # check local tool versions + cluster health

Full command reference, all 18 test-suite types, and DX commands (idp context inject, idp learn, idp mcp status, …): docs/cli-reference.md.

The Golden Path

Backstage → scaffold repo → push code
         → GitHub Actions CI (test + smoke-check)
         → GitHub Actions CD → ECR → EKS (Helm)   [AWS, on push to main]
         → idp:deploy-local (Backstage) → Kind     [local]
         → Prometheus ServiceMonitor → Grafana / CloudWatch

Scaffold a service or test suite via Backstage (http://backstage.idp.local → Create) or the idp CLI above. Deploy to Kind via Backstage's idp:deploy-local action, or helm upgrade --install my-svc ./helm/service-template .... Full walkthrough — template catalog, deploy steps, troubleshooting: docs/golden-path.md.

---

Roadmap

Status lives on the GitHub Project board and in the issues — that is the single source of truth. This section is the honest summary.

Recently shipped — Engineering Intelligence, all thirteen phases

The platform now scores its own engineering health. A framework-free scoring engine
(backstage/app/packages/engineering-intelligence-core) turns the telemetry four Python
exporters, the catalog, Tech Insights, OpenCost and Langfuse already produce into seven
dimension scores, each decomposing into evidence that names its metric, source and
timestamp. Served at /api/engineering-intelligence/*.

Three things the Phase 0 assessment found, which shaped the design:

• The Bronze/Silver/Gold scorecard is implemented three times and has already drifted —
  gold requires 9 passing checks in packages/app/src/scorecard.ts and 10 in
  observability/tech-insights-exporter/exporter.py, so a service can be Gold on its entity
  page and not Gold on the Grafana dashboard. The new engine consumes Tech Insights facts
  rather than becoming a fourth copy. Reconciling the existing three re-tiers live services
  and is tracked separately.
• There is no long-term metric store — Prometheus retained only 6h locally (since raised to 24h) and 30d on AWS, with
  no recording rules for any custom series. Snapshots are persisted from the first refresh
  because no history can be back-filled.
• Developer Experience had no data source at all. It reported insufficient-evidence
  with a null score rather than a number, and was excluded from the overall score rather
  than counted as zero. Phase 5 closed this, and the rule it established still governs every
  unmeasurable dimension. Security scores only what it can see — that scanning is
  declared — and says so in every evidence row.

On top of that, a five-level maturity model (Ad Hoc → Standardised → Platform Enabled →
AI Enabled → Autonomous) at /api/engineering-intelligence/maturity. Levels are floors
rather than an average, so one weak dimension holds the level down; a dimension with no
evidence makes the level unconfirmed rather than failed; and Level 5 declares two
requirements no collector supplies — enforced approval gating and measured agent
remediation — so it cannot be awarded from scores alone.

The Engineering Intelligence page at /engineering-intelligence renders all of it:
overall score, maturity ladder, seven dimension cards, the evidence behind any dimension,
and top risks. Each card links to the page that already owns its detail — /scorecard,
/slo, /finops, /dora, /langfuse — rather than redrawing those series. It is the
first custom frontend plugin here to live outside the 7,700-line extensions.tsx, and the
only page in the portal with no demo-data fallback: a failed request shows an error, not a
plausible-looking score.

Phases 4 and 5 closed the two biggest measurement gaps. Developer Experience is now
scored: the DORA exporter CronJob publishes devex_pr_cycle_time_hours,
devex_ci_duration_minutes and devex_build_failure_ratio from the workflow runs it
already fetches plus one bounded pull-request query — and omits a series rather than
pushing a zero when nothing merged or nothing ran. Platform Health gained a
/platform endpoint and a dashboard card: service and ownership counts, template usage,
scaffolder success rate, and the named services that are not on a golden path.

The two DORA exporters are a known drift pair that cannot share a module, so
observability/tests/test_dora_devex.py runs every assertion against both copies and
compares them directly — the first behavioural test these exporters have ever had, and
now a CI gate alongside the existing py_compile check.

Phase 6 adds a second scored model — AI Engineering Readiness across twelve areas at
/ai-readiness — reusing the same engine rather than reimplementing it. Six areas have a
collector (governance, evaluation, observability, model management via a new MLflow
registry collector, prompt management via Langfuse, and MCP reliability); six do not, and
say so. How many are measurable on a given install is lower again — a collector whose
source is not deployed reports nothing rather than guessing, so a platform without Langfuse
or MLflow sees three. AI architecture deliberately has no collector and never will: it is a judgement,
and a proxy for it would be the most dishonest number on the page.

Phase 7 turns "an evaluation suite exists" into "here is what it found". An extensible
evaluation model reads Langfuse scores and organises them by risk — correctness,
hallucination, PII safety, prompt injection, bias, regression — with one pattern table as
the extension point for a second evaluation library. Privacy, Security and Testing stop
being uncollectable as a result, though they still report insufficient evidence for an
organisation that runs no such suite: an untested risk is unknown, not absent.

Phase 8 attributes AI spend to teams — which the roadmap had recorded as blocked on a join
key that needed adding at the emitting end. It turned out one was already being written:
KAgent and MCP trace names carry the workload, and those names are catalog entities. Spend
whose name matches nothing is reported as an explicit unattributed remainder, never
redistributed across the teams that happen to be known. The scored signal is not how much
you spend but how much of the bill you can explain.

Phases 9–12 close it out. The AI Advisor answers leadership questions from the
structured reports, and its deliverable is the guardrails rather than a model call: a
sanitised context that drops evidence labels and raw trace names, and a mechanical check
that every claim cites a metric actually present. Asked which teams need attention, it
answers that Engineering Health is platform-wide and cannot rank teams — that refusal is
the feature. Executive reporting splits what improved from what declined and reports
no trend until two snapshots exist. Benchmarking ships the data model and anonymity
floor and transmits nothing — consent and custody are product decisions that precede
code. Multi-tenancy names the hierarchy a hosted deployment would need, with
single-tenant as the one-organisation case rather than a separate path, and no artificial
limits anywhere.

Running it against a real cluster then found the failure mode unit tests structurally
cannot: an upstream exporter publishing 0.0 where it should publish nothing. A repo
that had never deployed scored 100 for reliability, because a banded normaliser reads a 0%
change-failure rate as elite; unattributed spend scored as perfect budget discipline for the
same reason. Both are now withheld at the collector. A third case was the summary itself —
an "AI Readiness 97 / 100" derived from one measurable area out of twelve — so a headline
score is withheld below a third of its model, and the page says why rather than showing a
bare dash. Every one of these corrections moved a number down, which is the expected
direction when the previous figure was borrowing confidence from data that did not exist.

Quality Engineering was the last dimension with no signal, and the cause was not the
scaffolder templates — those already publish JUnit XML. It was that the only catalogued
repository with active CI is this one, and it uploaded a coverage profile with no per-test
outcomes. Its three test jobs now publish JUnit as test-results-*, and the flaky-test
exporter matches artifact names on prefix and reads every match, because
upload-artifact@v4 forbids two artifacts sharing a name in one run and taking only the
first would let one language's failures pass unseen.

Design decisions in ADR-0006; the phase
plan and what each one could and could not measure in
the roadmap; the collectors and their failure
behaviour in integrations.

Recently shipped — closing platform gaps

• LiteLLM as the model backend — Anthropic and Amazon Bedrock behind one endpoint, with virtual keys and per-team spend (ADR-0008).
• Consent/delegation gate and verified AI Assistant identity — the user an agent acts for is now verified server-side instead of trusted from a browser header, and mutating MCP tools check that the user has consented to that agent acting for them (Agent Approvals).
• Compliance watcher — when a scorecard check flips from passing to failing, the owner gets a Slack message and a Jira ticket (Shift-Left Leadership).
• One scorecard predicate package — @internal/scorecard-core is now shared by the entity-page tab and the Tech Insights retriever, which had drifted on three AI checks.
• Infra templates register their Resource entity — the 8 PR-based infra templates now tell you how to register the generated catalog-info.yaml after merge, and write into scoped directories instead of the repo root.
• GitHub Org Teams as the identity source — catalog groups and sign-in come from GitHub Teams, and new service repos default to the Teams org (ADR-0004).
• 24h local Prometheus retention (was 6h), so local trend views have more than a few hours of history.
• No more AI Assistant hangs — every KAgent agent is now told not to call ask_user, which the chat UI cannot answer.

Recently shipped — pre-open-source hardening

The AWS path went effectively untested between May and August 2026. Bringing a real cluster up surfaced a run of defects that are now fixed in the scripts, Terraform and manifests rather than worked around:

• The Terraform backend was pinned to the maintainer's own S3 bucket, so any other user's first terraform init failed ~30 seconds into a 40-minute script. Both modules now use a partial backend generated by setup.sh.
• Alertmanager's route tree terminated every alert before it reached the agent event router, so no incident record had ever been created automatically.
• The scaffolder was dead on AWS — Datadog APM's NODE_OPTIONS replaced rather than appended to the image's, dropping --no-node-snapshot.
• Guest auth was enabled in the production config, under a comment claiming it was not committed.
• 71 template files hardcoded *.idp.local, so every service scaffolded on AWS got catalog links that only resolve on a laptop.
• CI reported green on paths that ran no jobs at all, including two services with full test suites.

Design decisions from that work are recorded as ADRs rather than left implicit: batch orchestration, delivery model, incident management, identity and access, LLM serving and agent frameworks, Engineering Intelligence, AI Gateway, LiteLLM multi-provider gateway.

Known limitations

Stated plainly, because finding these by surprise is worse than reading them here:

Limitation	Detail
Coarse authorization	Any authenticated user can run any of the 64 templates against any namespace — GitHub Org Team sync (below) gates sign-in, not template execution. ADR-0004, issues #153 and #155
Sloth has no in-cluster operator	SLO rules are vendored; editing a source file without the sloth binary silently changes nothing
No CI exercises an AWS bootstrap	terraform validate and a guard against committed account ids is all that gates it

Next

Multi-team production hardening and self-hosted small-model serving. See the board. (The LangGraph agent template shipped — it is one of the 12 blessed templates. Amazon Bedrock shipped too, via LiteLLM — ADR-0008.)

---

Known Issues (local development)

Issue	Workaround
/kubernetes standalone page crashes	By design — disabled in local config. Use the Kubernetes tab on any catalog entity instead
Cost Overview shows "OpenCost returned 500"	Wait for the OpenCost pod: kubectl get pods -n opencost
Catalog empty on first load	Fixed: dangerouslyDisableDefaultAuthPolicy: true prevents a 401 flash before sign-in
ImagePullBackOff after scaffold	Image hasn't been pushed to the local registry yet. See docs/runbooks/image-pull-backoff.md
Backstage K8s tab shows "unknown" for CPU/memory	metrics-server not running (auto-installed by bootstrap-local.sh)

Known Issues (AWS)

Issue	Workaround
terraform init fails with AccessDenied or "Backend configuration required"	terraform/backend.hcl has not been generated. Run ./scripts/setup.sh, or let bootstrap.sh create it on first run
Nodes fail with NodeCreationFailure: Instances failed to join the kubernetes cluster ~20 min in	Usually a VPC/quota issue, or a cold-start apply that reached EKS without the NAT route. bootstrap.sh targets module.vpc alongside module.eks to prevent the latter
Error acquiring the state lock	An interrupted apply left a stale lock: cd terraform && terraform force-unlock <lock-id>
Scaffolder tasks fail with "requires --no-node-snapshot"	NODE_OPTIONS in the deployment replaced the image's value instead of appending. Fixed — it must contain both --no-node-snapshot and --require dd-trace/init
Tearing down leaves resources behind	Use ./scripts/cleanup.sh, not terraform destroy — orphaned ALBs hold the subnets Terraform is trying to delete

Why each of these was possible, and which file now prevents it:
docs/aws-install-failure-modes.md.

---

Working on this repo with Claude Code

This repo ships its own Claude Code configuration under
.claude/, so an agent working here starts with the platform's conventions rather than
re-deriving them. Nothing here is required to run the platform — it only affects how Claude
behaves inside this repo.

Skill (/name)	Use it for
platform-architect	Deciding where a change belongs — Terraform vs Crossplane vs Helm vs kubernetes/, which of the three interaction channels exposes a capability, which app-config layer
platform-engineer	Actually building the change across components; knows the per-component CI gate and runs it
platform-reviewer	Reviewing a diff against this repo's conventions (dual local/AWS coverage, both template front doors, accepted risks)
golden-path-steward	The 64 scaffolder templates and the idp CLI scaffolder that must stay in sync with them
qa-shift-left	Test strategy, the Bronze/Silver/Gold scorecard, contract testing, flaky-test quarantine
security-advisor	Kyverno/PSS, IRSA and least-privilege IAM, External Secrets, Dependabot triage
sre-responder	Live incidents, SLOs and burn-rate alerts, rollback, DR failover, postmortems

Two sub-agents back them for work that would otherwise flood the main context:
drift-detector (compares the known drift pairs — template skeleton vs CLI scaffolder,
app-config.yaml vs all-templates.yaml, local vs AWS Helm values) and platform-auditor
(sweeps a named domain against a checklist). Cross-cutting facts the skills share live in
.claude/context/platform-map.md; CLAUDE.md carries the always-loaded instructions.

---

Documentation

Doc	Description
Local Setup (Kind)	Full local walkthrough
AWS Deployment Guide	Step-by-step, pre-flight checklist, known issues
Golden Path	End-to-end scaffold → deploy → observe flow
Architecture	Deep-dive into each layer
CLI Reference	idp CLI commands and all 18 test-suite types
Scripts Reference	Every scripts/*.sh script
Multi-Region (V2)	Active-standby AWS across eu-central-1 + us-east-1
Team Management	Onboard a new team: namespace, SecretStore, ArgoCD, Grafana
AI Assistant	KAgent + MCP server setup and usage, plus Langfuse LLM observability and prompt versioning
ADR-0007: AI Gateway	One gateway (agentgateway) for MCP tool traffic and model calls — design rationale and consequences
ADR-0008: LiteLLM	LiteLLM behind the AI Gateway — Anthropic + Bedrock, virtual keys, spend tracking
Agentic Development Platform (ADP)	Agent-driven dev workflow + ops, HiTL approval gate, opt-in phases
Agent Approvals	HiTL gate for agent-initiated mutating actions — policy, approval API, Backstage UI
DORA + FinOps	DORA entity tab, SLOs, cost budgets
Engineering Intelligence	Engineering Health scoring, maturity model, the evidence contract, the collector integrations, the AI Advisor's guardrails, and the phase roadmap
Contract Testing	MCP-driven contract gates
Mobile Platform	Android / iOS / Flutter templates
Crossplane vs Terraform	When to use each
Security Scanning	SAST, DAST, SCA setup
Shift-Left Leadership	Bronze/Silver/Gold programme overview, plus the compliance watcher (Slack + Jira on a check regression)
Docker Recovery	Recover Kind after Docker Desktop restarts

Full docs site: moatazeldebsy.github.io/backstage-platform-template.

---

Contributing

Issues and PRs are welcome. Before opening a PR, run:

helm lint helm/service-template
cd backstage/app && yarn lint && yarn test
cd services/hello-service && go test ./...
cd cli && go build ./... && go vet ./...

---

License

MIT — free to use, fork, and build on.

---