🤖 Spring AI Integration

A curated, production-grade collection of Spring Boot modules demonstrating AI and LLM integrations — covering multi-provider chat, RAG pipelines, tool calling, MCP servers/clients, chat memory, multimodality, structured outputs, prompt engineering, observability, and much more.

📋 Table of Contents

• Overview
• Repository Structure
• Modules
  • Core Chat & LLM Integration
  • Prompt Engineering
  • Structured Output
  • Chat Memory
  • Tool Calling
  • Retrieval-Augmented Generation (RAG)
  • Multimodality
  • Model Context Protocol (MCP)
  • Observability & Optimization
  • Web Search
  • Docker Model Runner
• Prerequisites
• Getting Started
• Configuration
• HTTP Request Examples
• Technologies & Topics

🌟 Overview

Spring AI Integration is a hands-on, modular reference repository for developers who want to learn, explore, and build AI-powered applications using the Spring AI framework on top of Spring Boot.

Each sub-project is a self-contained Spring Boot application that showcases a specific Spring AI feature or integration pattern. The modules range from beginner-friendly basic chat completions all the way to advanced topics like Model Context Protocol (MCP) security, financial RAG pipelines, prompt caching, observability metrics, and Docker-based local model execution.

Why This Repository?

• Multi-provider support — OpenAI (GPT-4), Anthropic (Claude), and Ollama (local models) with easy provider swapping
• Production-focused patterns — advisors, memory management, vector stores, structured outputs, and prompt caching
• MCP ecosystem — full MCP server and client implementations across Stdio, WebFlux, and WebMVC transports
• No vendor lock-in — Spring AI's unified API abstracts away provider specifics
• Learning-first design — every module is focused, well-scoped, and independently runnable

📁 Repository Structure

Spring-AI-Integration/
│
├── spring-with-ai/                      # Introductory Spring AI basics
├── spring-ai-02-chat-with-llms/         # Chat with OpenAI (GPT-4)
├── spring-ai-03-chat-with-claude/       # Chat with Anthropic Claude
├── spring-ai-04-chat-with-ollama/       # Chat with local Ollama models
├── spring-ai-chat-options/              # Runtime chat configuration options
│
├── prompt-templates/                    # Prompt templating with variables
├── prompt-stuffing/                     # Prompt stuffing patterns
│
├── structured-output/                   # Bean/Map structured LLM outputs
├── native-structured-output/            # Native structured output (JSON mode)
│
├── chat-memory/                         # In-memory and persistent chat history
├── compacting-chat-memory-advisor/      # Memory compaction with advisors
│
├── tool-calling/                        # Function / tool calling integration
│
├── spring-ai-rag-vector-store/          # RAG with vector store (PGVector/simple)
├── spring-ai-financial-rag/             # Financial domain RAG pipeline
├── prompt-stuffing/                     # In-context document injection
│
├── multimodality/                       # Image + text multimodal inputs
│
├── mcp-server-stdio/                    # MCP Server via standard I/O
├── mcp-server-webflux/                  # MCP Server via WebFlux SSE
├── mcp-client-stdio/                    # MCP Client for stdio transport
├── spring-ai-mcp-client/                # Spring AI MCP client integration
├── spring-io-mcp-server/                # Spring.io MCP server example
├── spring-ai-mcp-elicitation/           # MCP elicitation patterns
├── spring-ai-mcp-security/              # Secured MCP with OAuth2/JWT
│
├── spring-ai-metrics/                   # Observability & Micrometer metrics
├── spring-ai-prompt-caching/            # Prompt caching for cost/latency
├── spring-ai-web-search/                # Web search tool integration
├── docker-model-runner/                 # Docker-based local model runner
│
└── http-requests.http                   # HTTP client sample requests

📦 Modules

Core Chat & LLM Integration

spring-with-ai

The entry point into Spring AI. Demonstrates basic ChatClient usage, autoconfiguration, and simple request/response patterns. Ideal starting point for beginners.

spring-ai-02-chat-with-llms

Chat with OpenAI GPT-4 using Spring AI's ChatClient API. Shows how to configure the OpenAI starter, send prompts, handle responses, and stream tokens.

Key concepts: OpenAiChatModel, ChatClient, Flux<String> streaming, system/user message roles.

spring-ai-03-chat-with-claude

Chat with Anthropic Claude (claude-3-5-sonnet / claude-3-haiku). Demonstrates Spring AI's Anthropic integration including prompt configuration and response handling.

Key concepts: AnthropicChatModel, multi-turn conversation, system prompt configuration.

spring-ai-04-chat-with-ollama

Chat with locally running LLMs via Ollama (e.g., Llama 3, Mistral, Phi-3). Zero cloud dependency — everything runs on your machine.

Key concepts: OllamaChatModel, local inference, Ollama Docker container setup.

spring-ai-chat-options

Demonstrates runtime configuration of chat parameters — temperature, top-p, max tokens, frequency penalty, etc. — both at startup and per-request level.

Key concepts: ChatOptions, OpenAiChatOptions, OllamaChatOptions, per-call overrides.

Prompt Engineering

prompt-templates

Shows how to use Spring AI's PromptTemplate with parameterized variables, allowing dynamic prompt construction from templates and input maps.

Key concepts: PromptTemplate, Message types, variable interpolation, system vs. user templates.

prompt-stuffing

Demonstrates the prompt stuffing pattern — injecting external document content directly into the prompt context rather than using a vector store, useful for smaller documents or quick prototyping.

Key concepts: Document content injection, context window usage, in-context retrieval.

Structured Output

structured-output

Shows how to extract structured Java objects (POJOs, records, Maps, Lists) from LLM responses using BeanOutputConverter, MapOutputConverter, and ListOutputConverter.

Key concepts: OutputConverter, BeanOutputConverter<T>, format instructions, JSON parsing.

native-structured-output

Uses native JSON mode (where supported by the provider) for guaranteed-valid JSON output from the LLM, bypassing prompt-based format instructions.

Key concepts: responseFormat, native JSON mode (OpenAI structured outputs), schema enforcement.

Chat Memory

chat-memory

Implements conversation memory to maintain chat history across turns. Covers both in-memory (for development) and persistent storage strategies.

Key concepts: MessageChatMemoryAdvisor, InMemoryChatMemory, ChatMemory, conversationId.

compacting-chat-memory-advisor

Demonstrates how to handle long conversations using a memory compaction advisor that summarizes older messages when the context window limit approaches.

Key concepts: AbstractChatMemoryAdvisor, compaction strategy, token-aware summarization.

Tool Calling

tool-calling

Full example of Spring AI's function/tool calling — registering Java methods as callable tools that the LLM can invoke during a conversation to fetch real-time data or execute logic.

Key concepts: @Tool, FunctionCallback, FunctionCallbackWrapper, tool registration, result handling.

@Bean
public FunctionCallback weatherFunction() {
    return FunctionCallbackWrapper.builder(new WeatherService())
        .withName("getWeather")
        .withDescription("Get the current weather for a given city")
        .withInputType(WeatherRequest.class)
        .build();
}

Retrieval-Augmented Generation (RAG)

spring-ai-rag-vector-store

Full RAG pipeline implementation: document ingestion, chunking, embedding generation, vector store persistence, and similarity-based retrieval at query time.

Key concepts: VectorStore, SimpleVectorStore, TokenTextSplitter, EmbeddingModel, QuestionAnswerAdvisor, document readers (PDF, text).

Architecture:

Document → Splitter → EmbeddingModel → VectorStore
                                            ↓
User Query → EmbeddingModel → Similarity Search → Retrieved Chunks
                                            ↓
                               ChatClient + Context → LLM → Answer

spring-ai-financial-rag

A domain-specific RAG application focused on financial documents. Ingests financial reports, filings, or market data and enables natural language Q&A over the content.

Key concepts: Domain-specific chunking strategies, finance-tuned prompts, retrieval confidence, source attribution.

Multimodality

multimodality

Demonstrates vision + text multimodal capabilities — sending images alongside text prompts to multimodal models (e.g., GPT-4o, Claude 3, LLaVA via Ollama).

Key concepts: UserMessage with media attachments, Media type, image URL and base64 inputs, vision model configuration.

UserMessage userMessage = new UserMessage(
    "Describe what you see in this image.",
    List.of(new Media(MimeTypeUtils.IMAGE_PNG, imageResource))
);

Model Context Protocol (MCP)

Spring AI Integration provides a comprehensive set of MCP modules covering server implementations, client integrations, security, and advanced patterns.

mcp-server-stdio

A Stdio-transport MCP server — communicates with the client via standard input/output streams. Ideal for local tool use with AI assistants like Claude Desktop.

Key concepts: StdioServerTransport, tool registration, MCP spec compliance.

mcp-server-webflux

A reactive MCP server using WebFlux SSE (Server-Sent Events) transport — suitable for HTTP-based, cloud-deployed MCP deployments.

Key concepts: WebFluxSseServerTransport, reactive streams, SSE endpoint, MCP tool exposure.

mcp-client-stdio

An MCP client that connects to a Stdio-based MCP server and invokes its registered tools through the Spring AI chat flow.

Key concepts: StdioClientTransport, McpSyncClient, tool discovery, function callback bridging.

spring-ai-mcp-client

A full Spring AI MCP client integration using the high-level Spring AI abstractions — connects to any MCP-compatible server and exposes its tools automatically to the ChatClient.

Key concepts: McpFunctionCallback, auto-tool-registration, Spring Boot autoconfiguration for MCP.

spring-io-mcp-server

An MCP server modeled after the Spring.io content structure — exposes tools for querying Spring ecosystem resources, projects, and documentation.

spring-ai-mcp-elicitation

Demonstrates the MCP elicitation pattern — the server proactively requests additional information from the user/client during a tool call.

Key concepts: Elicitation requests, dynamic input prompting, conversation-aware tool calls.

spring-ai-mcp-security

Implements OAuth2 / JWT-secured MCP — demonstrates how to protect MCP server endpoints with Spring Security, requiring proper bearer token authentication from MCP clients.

Key concepts: Spring Security OAuth2, JWT validation, SecurityFilterChain, protected tool endpoints.

Observability & Optimization

spring-ai-metrics

Integrates Micrometer observability into Spring AI — tracking token usage, latency, model calls, and errors via meters and traces. Compatible with Prometheus, Grafana, and Zipkin.

Key concepts: ObservationRegistry, ChatClientObservation, custom metrics, Spring Boot Actuator, Micrometer.

spring-ai-prompt-caching

Demonstrates prompt caching (supported by Anthropic Claude and other providers) to reduce latency and API cost when the same system prompt or context is reused across requests.

Key concepts: Cache control headers, Anthropic cache_control API, cost optimization, cache hit/miss metrics.

Web Search

spring-ai-web-search

Integrates real-time web search as a tool available to the LLM — allowing the model to fetch up-to-date information from the internet during a conversation.

Key concepts: Web search tool registration, search result injection, citation handling, Brave Search / Tavily integration.

Docker Model Runner

docker-model-runner

Shows how to use Docker's built-in Model Runner (available in Docker Desktop 4.40+) to run LLMs locally via a Docker-native endpoint, bypassing the need for a separate Ollama installation.

Key concepts: Docker Model Runner endpoint, spring.ai.openai.base-url override, local model execution, zero-dependency local AI.

🔧 Prerequisites

🚀 Getting Started

1. Clone the Repository

git clone https://github.com/drissiOmar98/Spring-AI-Integration.git
cd Spring-AI-Integration

2. Set Up API Keys

Create a .env file or export environment variables:

# OpenAI (GPT-4, embeddings)
export OPENAI_API_KEY=sk-your-openai-key

# Anthropic (Claude)
export ANTHROPIC_API_KEY=sk-ant-your-anthropic-key

3. Start Local Infrastructure (if needed)

For modules using Ollama:

docker run -d -p 11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama pull llama3

For modules using PGVector (RAG):

docker run -d \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=postgres \
  -e POSTGRES_DB=vectordb \
  -p 5432:5432 \
  pgvector/pgvector:pg16

4. Run a Module

Navigate to any module and start it:

cd spring-ai-02-chat-with-llms
./mvnw spring-boot:run

Or build and run the JAR:

./mvnw clean package -DskipTests
java -jar target/*.jar

⚙️ Configuration

Each module has its own application.properties or application.yml. Common configuration patterns:

# OpenAI
spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-4o
          temperature: 0.7

# Anthropic
spring:
  ai:
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        options:
          model: claude-3-5-sonnet-20241022

# Ollama (local)
spring:
  ai:
    ollama:
      base-url: http://localhost:11434
      chat:
        options:
          model: llama3

# Vector Store (PGVector)
spring:
  ai:
    vectorstore:
      pgvector:
        index-type: HNSW
        distance-type: COSINE_DISTANCE
        dimensions: 1536

🌐 HTTP Request Examples

The root http-requests.http file contains ready-to-use REST client examples for all modules. These can be run directly in IntelliJ IDEA or VS Code with the REST Client extension.

### Chat with OpenAI
POST http://localhost:8080/api/chat
Content-Type: application/json

{
  "message": "What is Spring AI?",
  "conversationId": "session-1"
}

### RAG Query
POST http://localhost:8080/api/rag/query
Content-Type: application/json

{
  "question": "What were the Q3 financial results?"
}

### Tool Calling
POST http://localhost:8080/api/chat/tools
Content-Type: application/json

{
  "message": "What is the weather like in Paris right now?"
}

### Multimodal (image + text)
POST http://localhost:8080/api/multimodal
Content-Type: application/json

{
  "message": "Describe this chart",
  "imageUrl": "https://example.com/chart.png"
}

🛠️ Technologies & Topics

GitHub Topics: java spring-ai springboot llms rag mcp mcp-server mcp-client mcp-security tool-calling prompt-engineering advisors multimodality structured-output vector-stores embedding open-ai ollama web-search docker-model-runner

Built with ❤️ using Spring AI — the portable, provider-agnostic AI framework for Java developers.