context-engineering

mcp
SUMMARY

[WIP] Context engineering: the art and science of shaping context-aware AI systems

README.md

Context engineering

Context engineering can be defined as the practice of designing systems that provide a Large Language Model (LLM) with all the necessary information to complete a task effectively. It goes beyond prompt engineering since it focuses on building a comprehensive and structured context from various sources like system instructions, external knowledge, memory, tools, and state. The central idea is that the success of a complex LLM-based system depends more on the quality and completeness of the context provided than on the specific wording of the prompt itself.

Tobi Lütke, the CEO of Shopify, coined the term context engineering in a tweet on June 19, 2025. He defined context engineering as the art of providing all the context for the task to be plausibly solvable by the LLM. This novel concept captures the essence of the current evolution of LLM-based systems, inspiring others (like me) to understand and define this emerging engineering discipline. Since then, I've been working on a book entitled Context engineering: the art and science of shaping context-aware AI systems, to be published by Manning in 2026.

This GitHub repository is intended to be a companion resource for this book and a go-to reference for practitioners looking to understand and adopt the context engineering principles.

Warning: This repository is a work in progress, so content and structure may change.

Table of contents

This book aims to provide a strong, general-purpose theoretical foundation for context engineering, supported by hands-on examples. Its table of contents is the following:

  1. Introduction to context engineering
  2. System instructions and user prompts
  3. Retrieval and external knowledge
  4. Tools and memory for AI agents
  5. State and multi-agent systems
  6. Context management and evaluation
  7. Context in AI frameworks
  8. Context engineering in real-world environments
  9. Context engineering through the software development lifecycle
  10. State of the art on context engineering
    Appendix A. The AI ecosystem
    Appendix B. References and further reading

Each chapter of this book starts by explaining the underlying principles and patterns of each thematic block. Then, the final part of each chapter is devoted to presenting specific examples. This GitHub repository contains all these examples. Moreover, I will include new examples and maintain the existing ones even after the book is published. The goal is to provide an open-source, updated reference for everyone interested in context engineering.

Online tool

This repo also hosts the context-aware prompt builder, an online tool presented in chapter 2 for designing, comparing, and reusing structured prompts across multiple frameworks and AI models.

Resources

Although the concept of context engineering is new, the underlying technologies (LLMs, AI agents, prompt engineering, RAG, MCP, memory management, skills, etc.) have been developed over the years. Nevertheless, summarizing all these converging technologies and tools in a single book is a very challenging task. As you know, nowadays, there is more information than ever, and it is very easy to get lost with so many sources. This section summarizes some of the most relevant references and resources I found during my journey to unravel the essence of context engineering.

Context Engineering

Prompt Engineering

  • Prompt Engineering (Lee Boonstra, 2025) Overview of prompt engineering principles and methods aimed at helping developers construct effective prompts for LLMs.
  • Prompt Engineering Guide (PromptingGuide.ai, 2025) Guide that defines prompt engineering as a discipline for designing and optimizing prompts to make LLMs perform better on diverse tasks, offering techniques, references, and best practices.
  • The Prompt Engineering Playbook for Programmers (Addy Osmani, 2025) Practical playbook targeting programmers, giving structured guidelines and patterns for writing prompts that yield consistent and reliable outputs from AI.
  • Anthropic's Prompt Engineering Interactive Tutorial (Anthropic, 2025) Hands-on tutorial that walks users step-by-step through designing and refining prompts (with exercises and example-based practice) to get better results from their models.
  • Meta's prompt engineering guide (Meta, 2025) Guide describing how to craft prompts for their models, covering best practices to structure instructions, context, and examples for improved performance.
  • Google's Gemini prompt engineering guide (Google, 2025) How to write effective prompts for Google's Gemini (or related models), emphasizing clarity, context, and structured prompting for reliable responses.
  • Prompt examples, by OpenAI (OpenAI, 2025) Collection of sample prompts illustrating good practices and giving real-world examples to help users understand how to frame prompts effectively for best results.
  • Prompt Library, by Anthropic (Anthropic, 2025) Repository of tested prompt templates and examples, serving as a practical library for users to build on and adapt for their own AI tasks.
  • Prompt Engineering Guide (DAIR.AI, 2026) Extensive guide covering prompt engineering techniques, examples, patterns, and best practices for working with LLMs.
  • Brex's prompt engineering guide (Brex, 2026) Practical prompt engineering guide from Brex, including patterns, examples, and structured instructions for improving LLM output quality.

AI Foundations (LLMs, Machine Learning)

  • AI Engineering: building applications with foundation models (Chip Huyen, Dec 04, 2024) Overview of how to build real-world applications using foundation models covering model selection, evaluation, prompt engineering, RAG, agents, deployment, and best practices for reliability and scalability.
  • The 2026 Guide to Machine Learning (IBM, 2026) Comprehensive guide covering the latest techniques, tools, and best practices in machine learning, including deep learning, LLMs, and AI system design.
  • Generative AI for beginners (Microsoft, 2026) Beginner-friendly curriculum teaching the fundamentals of generative AI, including notebooks, examples, and guided lessons.
  • Awesome Neuron (Awesome Neuron, 2025) Newsletter/blog exploring recent developments, tools and ideas around neural-agent frameworks and agentic AI.
  • Best AI and LLM Engineering Resources (Javin Paul, 2026) Large curated repository of high-quality resources on AI engineering, LLM workflows, prompt design, and system patterns.
  • Hands-On Large Language Models (Jay Alammar & Maarten Grootendorst, 2024) Official code repository for the O'Reilly book, featuring practical guides on using LLMs for various NLP tasks, fine-tuning, and advanced retrieval (RAG).
  • LLM Course (Maxime Labonne, 2024) Comprehensive roadmap and collection of Colab notebooks for mastering LLMs, divided into Fundamentals, building, and deploying.
  • AI agents for beginners (Microsoft, 2024) 10-lesson curriculum designed to teach the basics of building AI agents, covering architectures, tools, and multi-agent systems.
  • ML For Beginners (Microsoft, 2021) 12-week, 26-lesson curriculum offering a comprehensive introduction to "classic" machine learning using Scikit-learn.
  • GenAI Agents (Nir Diamant, 2024) Collection of tutorials and implementations for Generative AI agents, ranging from basic conversational bots to complex multi-agent workflows.
  • AI Engineering (Chip Huyen, 2025) AI engineering principles, covering data pipelines, LLM systems, deployment, evaluation, and production readiness.
  • AI Engineering Toolkit (Sumanth077, 2026) Collection of tools, templates, and best practices to support building, testing, and deploying AI/LLM-powered applications.
  • Awesome generative AI guide (Aishwaryanr, 2026) Curated set of learning resources, papers, tools, and tutorials for understanding and applying generative AI.
  • Awesome LLM Apps (Shubham Saboo, 2026) Curated list of real-world LLM application examples demonstrating practical use cases and design patterns.

AI Agents

  • Introduction to Agents, by Alan Blount et al. (Alan Blount et al., 2025) Definition of agent as a complete application, combining LLM reasoning, tooling and orchestration that plans and acts autonomously rather than just responding to single prompts.
  • What are AI Agents? Why do they matter? (Addy Osmani, 2025) Explanation that AI agents are programs that use language models and tools to perform goal-driven tasks, emphasizing their importance for automating complex workflows.
  • Agents Companion, by Antonio Gulli et al. (Antonio Gulli et al., 2025) Whitepaper presenting production-grade considerations for agents: orchestration, memory, evaluation, multi-agent coordination and deployment best practices.
  • The AI agents stack (Letta, 2025) Overview of the technological layers and components (models, tools, orchestration, memory, environment interfaces) that constitute a full-fledged AI-agents infrastructure.
  • Gemini CLI Tips & Tricks (Addy Osmani, 2025) Practical advice and examples on using the Gemini CLI tool effectively when building or interacting with AI agents.
  • Memory by LangGraph (LangGraph, 2025) Guide about memory management techniques and how to maintain agent state across interactions to support coherent, long-lived agent sessions.
  • Gemini with memory (Phil Schmid, 2025) Tutorial showing how to integrate persistent memory mechanisms into agents using Gemini, enabling context-aware, multi-step workflows.
  • Agent Quality (Meltem Subasioglu et al., 2025) Research-oriented paper evaluating metrics and benchmarks for assessing agent performance, robustness and reliability across tasks.
  • Open Source LLM Tools (Huyenchip, 2025) Curated list and analysis of open-source tools for building, evaluating or deploying LLM-based agents.
  • The AI agents stack (Letta, 2025) Article outlining the architecture layers (model, tools, memory, orchestration) that build up modern AI-agent systems.
  • AG-UI: Agents to users (AG-UI community, 2025) Open-source project aiming to provide user-facing UI/UX for agent-based systems.
  • AI Agents in Action (Micheal Lanham, 2025) Practical guide to designing, building and deploying LLM-powered autonomous agents and multi-agent systems, including memory, tool integration, orchestration, and real-world use cases.
  • Building AI Agents with LLMs, RAG, and Knowledge Graphs (Salvatore Raieli & Gabriele Iuculano, Jul 2025) Step-by-step book showing how to combine LLMs with RAG and knowledge graphs to build agents capable of grounded reasoning, tool use, planning and complex task execution.
  • AI Agents: The Definitive Guide (Nicole Koenigstein, 2025) Designing, evaluating, and deploying AI agents; covers architecture, memory, tools, orchestration and production-grade considerations.
  • An Illustrated Guide to AI Agents (Maarten Grootendorst, Jay Alammar, 2025) Conceptual guide that explains agents, memory, tool use, workflows for agent-based AI.
  • Agents Towards Production (Nir Diamant, 2026) Practical resources and examples demonstrating how to take AI agents from experimentation to reliable production systems.

MCP

Retrieval and RAG

  • Introduction to Information Retrieval (Christopher Manning et al., 2008) Classic textbook covering foundational concepts in information retrieval, indexing, search algorithms, ranking, etc.
  • Introducing Contextual Retrieval (Anthropic, Sep 19, 2024) Introduction of Contextual Retrieval, a method that enriches document chunks with additional context before embedding/indexing to improve accuracy and reduce context loss in RAG systems.
  • Chunking Strategies for LLM Applications, by Pinecone (Pinecone, 2025) Guide explaining how to break large texts into appropriately sized chunks before embedding, a critical step for effective retrieval and generation with LLM-based applications.
  • Retrieval, by LangChain (LangChain, 2025) Reference documentation on retrieval functionality in LangChain, describing how to integrate vector-based retrieval into LLM workflows.
  • What is Agentic RAG (Weaviate, 2025) Explanation of “Agentic RAG,” a paradigm that combines retrieval-augmented generation with autonomous agents, enabling multi-step reasoning, tool use, and dynamic retrieval for complex workflows.
  • Thinking Beyond RAG: Why Context-Augmented Generation Is Changing the Game (Yusuf Ishola, 2025) Explanation of Context-Augmented Generation and how it improves over traditional RAG.
  • RAG vs CAG vs Fine-Tuning (Rafa Paez, 2025) Comparative analysis of three approaches: Retrieval-Augmented Generation (RAG), Contextual Augmented Generation (CAG), and fine-tuning, discussing trade-offs in accuracy, adaptability, cost, and maintenance for each.
  • RAG-Anything: All-in-One RAG Framework (Data Intelligence Lab@HKU, 2026) Open-source framework providing an all-in-one pipeline for RAG.
  • Agentic RAG for Dummies (Giovanni Pasquale, 2026) Beginner-friendly guide to understanding and implementing Agentic RAG, with practical examples and code snippets.

AI for software development

  • The AI-Native Software Engineer (Addy Osmani, Jul 01, 2025) Essay exploring what it means to be a software engineer in a world where AI is deeply integrated, focusing on new skills, responsibilities, and the evolving identity of engineers.
  • The reality of AI-Assisted software engineering productivity (Addy Osmani, Aug 16, 2025) Recent evidence showing that AI can accelerate parts of coding but often leaves the last 30%: debugging, maintenance, and architectural decisions, to humans, arguing the future is human and AI, not AI-alone.
  • Vibe coding is not the same as AI-Assisted engineering (Addy Osmani, Aug 30, 2025) Warns against conflating vibe coding (prompt-driven quick prototyping) with robust, production-ready AI-assisted engineering, highlighting differences in ownership, quality, and maintainability.
  • Coding for the Future Agentic World (Addy Osmani, 2025) Overview of how developers should adapt their workflows and mindset to build for an agentic world, where AI agents are first-class collaborators in software development.
  • Conductors to Orchestrators: The Future of Agentic Coding (Addy Osmani, Nov 01, 2025) Software engineers will shift from writing code (coder) to directing AI agents (conductor) and ultimately to overseeing fleets of agents (orchestrator), redefining the role of developers in the agent-driven future.
  • Generative AI for Software Development (Sergio Pereira, 2025) Generative AI and LLMs leveraged to assist software development, including coding, documentation, and planning, via agentic or semi-agentic patterns.
  • Beyond Vibe Coding: A practical guide to AI-assisted development (Addy Osmani, 2025) Practical handbook advocating for structured, engineering-style workflows (not ad-hoc prompts) when using AI in software development, emphasizing reproducibility, context management, and scalable patterns.

YouTube channels

  • Andrej Karpathy Deep dive lectures explaining AI from first principles, including building GPT models from scratch.
  • sentdex Practical Python programming tutorials covering machine learning, finance, robotics, and more.
  • Sebastian Raschka In-depth technical content on machine learning, deep learning, and building LLMs.
  • Jeremy Howard Focused on making deep learning accessible to coders through a top-down learning approach.
  • MIT OpenCourseWare Free, high-quality video lectures from MIT classrooms, including world-renowned AI courses.
  • Stanford Online Professional and graduate-level courses from Stanford University, including famous ML and NLP series.
  • StatQuest with Josh Starmer Breaking down complex statistics and machine learning topics into intuitive, bite-sized pieces.
  • 3Blue1Brown Visualizing mathematics and the "why" behind neural networks and linear algebra.
  • Krish Naik Comprehensive resource for data science, MLOps, and Generative AI with a focus on industry readiness.
  • CampusX Structured mentorship and bootcamps for Python, machine learning, and deep learning.

Papers

Contributing

If you think something should be improved or want to contribute to this repo, please open a pull request. Any comments or feedback are welcome.

About

context-engineering (Copyright © 2025-2026) is an open-source project created and maintained by Boni Garcia, licensed under the terms of Apache 2.0 License.

Reviews (0)

No results found