AI Engineering Playbook

Name: ai-engineering-playbook
Author: karthikreddy-7

A zero-to-100 learning path for applied AI engineering.

RAG · embeddings · vector search · agents · MCP · and the production engineering around them.

📖 karthikreddy-7.github.io/ai-engineering-playbook

Search, dark mode, diagrams, and the interlinked path — this repo is the source.

What this is

The consumer side of AI engineering: you take models you did not train and build systems that are
correct, fast, cheap, and safe enough to put in front of real users.

Most material on this subject is a pile of definitions. Definitions tell you what something is
called. They don't tell you what breaks without it, what it costs, or what you'd reach for instead
— and those are the things that matter when you're the one building it.

So every page answers three questions in order: why does this exist, how does it actually
work, and what goes wrong in production.

This is not a machine-learning course. No PyTorch, no training loops, no backprop. Model
internals appear only at survival depth — enough to reason about a boundary question without
bluffing.

Who it's for

Engineers building at the application layer who want to understand these systems properly rather
than assemble them from tutorials. It assumes you can code and have called an LLM API.

It's organized around the questions that decide whether a system works:

How does RAG actually work — and what does productionizing it involve?
How does HNSW work, and why cosine similarity?
How do you evaluate retrieval quality when there's no single right answer?
What does document Q&A over 10M documents for 50k users look like?
When should you not build an agent?
Where do latency, cost, and failures come from in an LLM pipeline?

How every page is built

Section	What it does
Prerequisites	What the page assumes, linked — so you can walk up when something is shaky
The intuition	The idea in plain language, with one load-bearing analogy, before any terminology
Why it exists	The specific pain that makes the concept necessary, and why the alternatives lose
The core idea	The concept stated properly and densely — the version you'd want to explain out loud
How it actually works	One level down, with a diagram: mechanism, parameters, and the maths that matter
The flows	The distinct end-to-end paths, each with what selects it and what breaks it
A worked example	One concrete input traced through with real numbers — then deliberately broken
Production concerns	Scale, cost, latency, failure modes, and what only shows up with real traffic
Common drill-downs	The questions a good engineer asks one level down, with answers
Test yourself	Scenario questions with collapsed answers — reasoning, not recall
Go deeper	Curated, individually verified links: official docs, canonical blogs, papers, talks
Where this connects	The natural next topics, and why you'd go there

Structure

Section	What it covers
01 — LLM Fundamentals	How LLMs work (survival depth), tokens & context, sampling, prompting, structured output, APIs, hallucination
02 — Embeddings & Search	Embeddings, similarity metrics, ANN indexes (HNSW/IVF), hybrid search, rerankers, embeddings beyond RAG
03 — Vector Databases	Choosing a vector DB, metadata filtering, scaling & operations
04 — RAG	The pipeline, chunking, retrieval patterns, graph & agentic RAG, evaluation, productionization
05 — Agents	The agent loop, tool calling, orchestration & memory, multi-agent, reliability, agents vs workflows
06 — MCP	The Model Context Protocol, and building MCP servers
07 — Production	Latency, cost, reliability, observability, security, evals & testing
08 — Model Landscape	Choosing models, open source & self-hosting, fine-tuning vs RAG
09 — Multimodal	Vision, audio, and multimodal ingestion at survival depth
10 — System Design	A repeatable reasoning framework + five worked designs
11 — Rapid Fire	Flashcard-style Q→A per section, plus tradeoff one-liners

56 topic pages · 11 sections · 5 worked system designs · 10 recall decks.

How to use it

Read the whole path — sections in nav order. Each assumes the ones above it, and pages inside a
section are ordered so nothing is used before it's introduced. Follow the
dependency map and
you'll never hit an idea you haven't met.

Repair one weak spot — start at the page you're shaky on and walk its Prerequisites links
upward until you hit something solid, then come back down.

Fast recall — the rapid-fire decks and
tradeoff one-liners. Cover the answer, say it out loud,
uncover, compare. Recognising an answer is not the same as being able to produce one.

The running example

One system recurs across the site so abstract ideas land on something concrete: a governed
enterprise LLM platform — a model gateway over a managed cloud model service, with
entitlement-filtered retrieval over internal documents, temporary document upload, grounded web
search, an agent mode calling internal tools over MCP, and a guardrails/observability layer around
all of it.

It's deliberately generic — an architecture pattern many organizations converge on, not any one
company's system. Numbers used with it are illustrative typical scale, not measured figures.

Correctness policy

Every external link was fetched and verified before being committed. No dead links, no
hallucinated citations.
mkdocs build --strict runs in CI, so a broken internal link, a missing anchor, or a page
left out of the nav fails the build before it can reach the site.
This field rots fast. Model names, prices, context limits, and API specifics change monthly.
Mechanisms and tradeoffs don't. Cross-check any specific figure against provider docs before
relying on it.

Running it locally

pip install -r requirements.txt
mkdocs serve          # http://127.0.0.1:8000
mkdocs build --strict # what CI runs

Contributing

Spotted an error, a stale claim, or a better canonical resource? Issues and PRs welcome — this
field moves fast and correctness beats coverage. If you're adding a reference link, please fetch
it first and confirm it says what the description claims.

License

MIT