Cloud Native AI Lab

Give a Kubernetes engineer a laptop and seven labs, and watch them schedule GPUs,
coordinate distributed training, and serve a model behind an autoscaling API.
No GPU required, no cloud bill.

Every lab runs on a local kind cluster. If you know pods, deployments, and
services, you can start at Lab 00 today. No machine learning background needed.

What you get
7 progressive labs	Kueue, DRA, JobSet, KServe, Gateway API Inference, kagent and MCP
Runs on a laptop	3-node `kind` cluster, about 8GB RAM, no cloud spend
72 automated tests	version pins, links, manifests, and security context are CI-gated (tests/)
11 project one-pagers	each dated and re-verified against upstream releases (docs/projects/)

Built as the companion resource to a Cloud Native University talk at
KubeCon + CloudNativeCon Europe 2026,
delivered 24 March 2026 at the RAI in Amsterdam, and maintained since as a
standing community resource. The
talk outline is the script as delivered.

Who is this for?

Kubernetes practitioners who are new to AI/ML workloads. If you know pods, deployments, services, and namespaces, you have everything you need to start. This lab does not assume any machine learning knowledge.

Learning path

┌─────────┐    ┌──────────┐    ┌─────────┐    ┌──────────┐    ┌──────────┐    ┌─────────────┐    ┌───────────┐
│  Setup  │───▶│  Kueue   │───▶│   DRA   │───▶│  JobSet  │───▶│  KServe  │───▶│ Gateway API │───▶│ kagent +  │
│ Lab 00  │    │  Lab 01  │    │ Lab 02  │    │  Lab 03  │    │  Lab 04  │    │   Lab 05    │    │ MCP Lab06 │
└─────────┘    └──────────┘    └─────────┘    └──────────┘    └──────────┘    └─────────────┘    └───────────┘
  Cluster        GPU job         Device        Distributed      Model          Inference          AI agents
  setup          scheduling      allocation    training         serving        routing            and tools

Labs

Lab	Title	What you'll learn
00	Environment Setup	Create a kind cluster configured for AI workload labs
01	Kueue Basics	Fair sharing, admission control, and preemption for GPU-like workloads
02	DRA Resource Claims	Declarative device allocation, and how Kubernetes replaces the old device plugin model
03	JobSet Training	Coordinated multi-pod training with failure handling
04	KServe Inference	Deploy a model, serve it behind an API, and watch it autoscale
05	Gateway API Inference Routing	Model-aware traffic routing with InferenceObjective and InferencePool
06	kagent and MCP	Deploy AI agents on Kubernetes with the Model Context Protocol

Optional: real GPU clusters

These labs create cloud resources that cost real money. Use them to run Labs 01-06 with actual GPUs.

Cloud	Lab
AWS EKS	labs/optional/eks-gpu-cluster
Google GKE	labs/optional/gke-gpu-cluster
Azure AKS	labs/optional/aks-gpu-cluster

Reference materials

One-page summaries of every project covered in the labs:

Project	What it does	Status
DRA	Declarative GPU allocation	Core K8s (GA in 1.34)
Kueue	Job queuing and fair sharing	CNCF Incubating
JobSet	Distributed training orchestration	kubernetes-sigs (v0.12.0)
LeaderWorkerSet	Multi-host inference topology	kubernetes-sigs (v0.9.0)
KServe	Model serving platform	CNCF Incubating (v0.19.0)
Knative	Scale-to-zero serverless	CNCF Graduated
llm-d	Distributed LLM inference	Launched by Red Hat (v0.8.1)
Gateway API Inference	Model-aware routing	GA (InferenceObjective + InferencePool)
kagent	Kubernetes-native AI agents	CNCF Sandbox
MCP	Model Context Protocol	AAIF / Linux Foundation
AAIF	Agentic AI Foundation	Linux Foundation

Full reports:

Cloud-Native AI Landscape Report: ecosystem overview, as researched in March 2026
Talk Outline: the 15-minute KubeCon EU 2026 talk, as delivered

Prerequisites

kind v0.32+
kubectl
Helm 3.x
~8GB RAM for the kind cluster
Docker installed and running

Quick start

git clone https://github.com/peopleforrester/cloud-native-ai-lab.git
cd cloud-native-ai-lab
cd labs/00-setup && ./setup.sh

Then follow the learning path checklist.

What works on kind vs. what needs GPUs

Feature	Works on kind?	Notes
Kueue fair sharing and preemption	Yes	Uses CPU/memory quotas
DRA resource claims	Partial	Conceptual walkthrough; optional DRA example driver
JobSet coordinated startup	Yes	Uses busybox to simulate training
KServe model serving	Yes	Small sklearn model on CPU
Gateway API inference routing	Partial	Annotated manifests; full demo needs GPU backends
kagent and MCP	Yes	CRD installation and resource creation

Contributing

See CONTRIBUTING.md. All work targets the staging branch.

Credit

Author: Michael Forrester, KodeKloud.

License

This project is licensed under the Apache License 2.0. See LICENSE for details.