MMSP - Model Message Stream Protocol

Integrate every model the same way, and keep one API in your head instead of one per provider.

MMSP, the Model Message Stream Protocol, takes the per-provider differences off a developer's mind: one message format and one streaming grammar for every model provider, in Python and TypeScript.

📖 Documentation: mmsp.penguin.ooo

Using a coding agent? Install the MMSP SKILL files from skills/ so it can use MMSP correctly in generated code.

📢 Follow us on or join our

Why MMSP?

• 🔗 Unified: A consistent and intuitive interface for developing agents across different LLMs.

• 🎯 Precise: Automatically handles interleaved thinking during multi-step tool calls, preventing performance degradation.

• 🧭 Traceable: Provides lightweight yet fine-grained tracing for debugging and auditing LLM executions.

Features

AutoLLMClient (Python & TypeScript)

Switch different LLMs with zero code changes and no performance loss.

Built-in Observability

Audit LLM executions by adding a single trace_id parameter, no database required.

https://github.com/user-attachments/assets/c49a21a1-5bf9-4768-a76d-f73c9a03ca87

Supported Models

Clients

AutoLLMClient takes a model id and a client_type. An official client speaks its
vendor's own API, knows the vendor's models, and reads the vendor's key from the environment;
a compatible client speaks one wire protocol for any endpoint that serves it.

client_type may be omitted for a model id that begins with a known family: gpt- and
text-embedding- route to openai-official, claude- to anthropic-official, gemini- to
gemini-official, glm- to zai-official, kimi- to moonshot-official, deepseek- to
deepseek-official, minimax- to minimax-official. Any other id raises and asks for a
client_type. The CLIENT_TYPE environment variable names one for every client the code does
not.

Gemini on Google Vertex AI takes the service-account JSON key as the API key; gemini-official
serves such a key through generateContent, because Vertex AI's Interactions endpoint serves none
of these models.

Where a gateway serves more than one protocol, prefer "openai-responses": OpenRouter serves it
for every model it hosts, while SiliconFlow serves Chat Completions only.

The full machine-readable list — model, base URL, client, input/output modalities, context
window, and per-million-token list pricing in USD or CNY:

from mmsp import list_supported_models

models = list_supported_models(currency="CNY")  # "USD" by default

import { listSupportedModels } from "@prismshadow/mmsp";

const models = listSupportedModels("CNY"); // "USD" by default

Installation

Python package

Install from PyPI:

uv add mmsp
# or
pip install mmsp

Build from source:

cd src_py && make

See src_py/README.md for comprehensive usage examples and API documentation.

TypeScript package

Install from npm:

npm install @prismshadow/mmsp

Build from source:

cd src_ts && make install && make build

See src_ts/README.md for comprehensive usage examples and API documentation.

Agent Skills

MMSP provides Codex/Claude Code skill files for assistants that need to help users consume the SDK packages:

• Python skill: skills/mmsp-python/SKILL.md
• TypeScript skill: skills/mmsp-typescript/SKILL.md

APIs

AutoLLMClient is the main class for interacting with the MMSP SDK. It is constructed with model and client_type (see Clients; a model id of a known family names its official client on its own), plus optional api_key, base_url, and default_headers — headers sent with every request, for endpoints that demand their own. It provides the following methods:

A key goes only where it was given for: a client reads OPENAI_API_KEY / ANTHROPIC_API_KEY from the environment only together with OPENAI_BASE_URL / ANTHROPIC_BASE_URL (or the provider's own endpoint), so a base_url passed in needs an api_key passed in with it, or the client raises at construction. A vendor client (deepseek-v4, glm-5.x, kimi-k*, minimax-m3, Gemini) reads its own variable, whatever endpoint it is given.

• (async) streaming_response(messages, config): Streams the response of LLMs in a stateless manner.
• (async) streaming_response_stateful(message, config): Streams the response of LLMs in a stateful manner.
• (async) list_models(): Lists the model ids the configured endpoint serves. A protocol client (openai-chat, openai-chat-vllm-adapter, openai-responses, ant-messages, openai-embedding) is named explicitly and lists everything the endpoint serves; a client deduced from a model id lists only the ids that deduce back to it.
• clear_history(): Clears the history of the stateful LLM client.
• get_history(): Returns the history of the stateful LLM client.
• set_history(history): Replaces the history of the stateful LLM client with a copy of the provided list.

Both streaming methods yield delta events, each carrying one content item, followed by exactly one stop event that carries the usage and the finish reason (see UniEvent).

Streaming clients skip output they do not recognize, so a gateway's own frames cannot end a generation. Set MMSP_DEBUG to anything other than 0, false, no or off to make it raise instead.

Basic Usage

[!NOTE]
We recommend using the stateful interface when calling the MMSP SDK.

OpenAI GPT-5.6

Python Example:

import asyncio
import os
from mmsp import AutoLLMClient

os.environ["OPENAI_API_KEY"] = "your-openai-api-key"

async def main():
    client = AutoLLMClient(model="gpt-5.6-sol")
    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Say 'Hello, World!'"}]
        },
        config={"temperature": 1.0}
    ):
        print(event)

asyncio.run(main())
# {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': 'Hello'}], 'usage_metadata': None, 'finish_reason': None}
# {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': ','}], 'usage_metadata': None, 'finish_reason': None}
# {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': ' World'}], 'usage_metadata': None, 'finish_reason': None}
# {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': '!'}], 'usage_metadata': None, 'finish_reason': None}
# {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.done', 'text': 'Hello, World!'}], 'usage_metadata': None, 'finish_reason': None}
# {'role': 'assistant', 'event_type': 'stop', 'content_items': [], 'usage_metadata': {'cached_tokens': 0, 'prompt_tokens': 12, 'thoughts_tokens': 0, 'response_tokens': 8}, 'finish_reason': 'stop'}

TypeScript Example:

import { AutoLLMClient } from "@prismshadow/mmsp";

process.env.OPENAI_API_KEY = "your-openai-api-key";

async function main() {
  const client = new AutoLLMClient({ model: "gpt-5.6-sol" });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text.done", text: "Say 'Hello, World!'" }]
    },
    config: {}
  })) {
    console.log(event);
  }
}

main().catch(console.error);
// {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': 'Hello'}], 'usage_metadata': null, 'finish_reason': null}
// {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': ','}], 'usage_metadata': null, 'finish_reason': null}
// {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': ' World'}], 'usage_metadata': null, 'finish_reason': null}
// {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.delta', 'text': '!'}], 'usage_metadata': null, 'finish_reason': null}
// {'role': 'assistant', 'event_type': 'delta', 'content_items': [{'type': 'text.done', 'text': 'Hello, World!'}], 'usage_metadata': null, 'finish_reason': null}
// {'role': 'assistant', 'event_type': 'stop', 'content_items': [], 'usage_metadata': {'cached_tokens': 0, 'prompt_tokens': 12, 'thoughts_tokens': 0, 'response_tokens': 8}, 'finish_reason': 'stop'}

Anthropic Claude Opus 5

import asyncio
import os
from mmsp import AutoLLMClient

os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-api-key"

async def main():
    client = AutoLLMClient(model="claude-opus-5")
    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Say 'Hello, World!'"}]
        },
        config={}
    ):
        print(event)

asyncio.run(main())

import { AutoLLMClient } from "@prismshadow/mmsp";

process.env.ANTHROPIC_API_KEY = "your-anthropic-api-key";

async function main() {
  const client = new AutoLLMClient({ model: "claude-opus-5" });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{"type": "text.done", "text": "Say 'Hello, World!'"}]
    },
    config: {}
  })) {
    console.log(event);
  }
}

main().catch(console.error);

OpenRouter GLM-5.3

import asyncio
import os
from mmsp import AutoLLMClient

os.environ["ZAI_API_KEY"] = "your-openrouter-api-key"
os.environ["ZAI_BASE_URL"] = "https://openrouter.ai/api/v1"

async def main():
    client = AutoLLMClient(model="z-ai/glm-5.3")
    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Say 'Hello, World!'"}]
        },
        config={}
    ):
        print(event)

asyncio.run(main())

import { AutoLLMClient } from "@prismshadow/mmsp";

process.env.ZAI_API_KEY = "your-openrouter-api-key";
process.env.ZAI_BASE_URL = "https://openrouter.ai/api/v1";

async function main() {
  const client = new AutoLLMClient({ model: "z-ai/glm-5.3" });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{"type": "text.done", "text": "Say 'Hello, World!'"}]
    },
    config: {}
  })) {
    console.log(event);
  }
}

main().catch(console.error);

SiliconFlow Qwen3.6 35B via OpenAI-compatible API

import asyncio
import os
from mmsp import AutoLLMClient

os.environ["OPENAI_API_KEY"] = "your-siliconflow-api-key"
os.environ["OPENAI_BASE_URL"] = "https://api.siliconflow.cn/v1"

async def main():
    client = AutoLLMClient(model="Qwen/Qwen3.6-35B-A3B", client_type="openai-chat")
    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Say 'Hello, World!'"}]
        },
        config={}
    ):
        print(event)

asyncio.run(main())

import { AutoLLMClient } from "@prismshadow/mmsp";

process.env.OPENAI_API_KEY = "your-siliconflow-api-key";
process.env.OPENAI_BASE_URL = "https://api.siliconflow.cn/v1";

async function main() {
  const client = new AutoLLMClient({
    model: "Qwen/Qwen3.6-35B-A3B",
    clientType: "openai-chat",
  });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text.done", text: "Say 'Hello, World!'" }],
    },
    config: {}
  })) {
    console.log(event);
  }
}

main().catch(console.error);

SiliconFlow Qwen3 Embedding 0.6B via OpenAI-compatible API

import asyncio
import os
from mmsp import AutoLLMClient

os.environ["OPENAI_API_KEY"] = "your-siliconflow-api-key"
os.environ["OPENAI_BASE_URL"] = "https://api.siliconflow.cn/v1"

async def main():
    client = AutoLLMClient(model="Qwen/Qwen3-Embedding-0.6B", client_type="openai-embedding")

    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Hello world"}],
        },
        config={},
    ):
        print(event)

asyncio.run(main())

import { AutoLLMClient } from "@prismshadow/mmsp";

process.env.OPENAI_API_KEY = "your-siliconflow-api-key";
process.env.OPENAI_BASE_URL = "https://api.siliconflow.cn/v1";

async function main() {
  const client = new AutoLLMClient({
    model: "Qwen/Qwen3-Embedding-0.6B",
    clientType: "openai-embedding",
  });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text.done", text: "Hello world" }],
    },
    config: {},
  })) {
    console.log(event);
  }
}

main().catch(console.error);

DeepSeek via the OpenAI Responses protocol

Any compatible endpoint can be called through the generic protocol clients by picking the
client type (openai-chat / openai-responses / ant-messages) and the provider's base
URL for that protocol:

import asyncio
import os
from mmsp import AutoLLMClient

async def main():
    client = AutoLLMClient(
        model="deepseek-v4-flash",
        api_key=os.environ["DEEPSEEK_API_KEY"],
        base_url="https://api.deepseek.com",
        client_type="openai-responses",
    )
    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Say 'Hello, World!'"}]
        },
        config={}
    ):
        print(event)

asyncio.run(main())

import { AutoLLMClient } from "@prismshadow/mmsp";

async function main() {
  const client = new AutoLLMClient({
    model: "deepseek-v4-flash",
    apiKey: process.env.DEEPSEEK_API_KEY,
    baseUrl: "https://api.deepseek.com",
    clientType: "openai-responses",
  });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text.done", text: "Say 'Hello, World!'" }],
    },
    config: {},
  })) {
    console.log(event);
  }
}

main();

DeepSeek via the Anthropic Messages protocol

import asyncio
import os
from mmsp import AutoLLMClient

async def main():
    client = AutoLLMClient(
        model="deepseek-v4-flash",
        api_key=os.environ["DEEPSEEK_API_KEY"],
        base_url="https://api.deepseek.com/anthropic",
        client_type="ant-messages",
    )
    async for event in client.streaming_response_stateful(
        message={
            "role": "user",
            "content_items": [{"type": "text.done", "text": "Say 'Hello, World!'"}]
        },
        config={}
    ):
        print(event)

asyncio.run(main())

import { AutoLLMClient } from "@prismshadow/mmsp";

async function main() {
  const client = new AutoLLMClient({
    model: "deepseek-v4-flash",
    apiKey: process.env.DEEPSEEK_API_KEY,
    baseUrl: "https://api.deepseek.com/anthropic",
    clientType: "ant-messages",
  });
  for await (const event of client.streamingResponseStateful({
    message: {
      role: "user",
      content_items: [{ type: "text.done", text: "Say 'Hello, World!'" }],
    },
    config: {},
  })) {
    console.log(event);
  }
}

main();

The same model works over client_type="openai-chat" (base URL
https://api.deepseek.com); OpenRouter, Z.AI, and MiniMax expose all three protocols the
same way.

Concepts: UniConfig, UniMessage and UniEvent

UniConfig

UniConfig is an object that contains the configuration for LLMs.

Example UniConfig:

{
  "max_tokens": 1024,
  "temperature": 1.0,
  "tools": [
    {
      "name": "get_current_weather",
      "description": "Get the current weather in a given location",
      "parameters": {
          "type": "object",
          "properties": {
              "location": {
                  "type": "string",
                  "description": "The city and state, e.g. San Francisco, CA"
              }
          },
          "required": ["location"]
      }
    }
  ],
  "thinking_summary": true,
  "thinking_level": "none | low | medium | high | xhigh | max",
  "tool_choice": "auto | required | none | a list of allowed tool names",
  "system_prompt": "You are a helpful assistant.",
  "prompt_caching": "enable | disable | enhance",
  "fast_mode": false,
  "image_config": {"aspect_ratio": "4:3", "image_size": "1K"},
  "tts_config": [{"voice": "Kore"}],
  "embedding_config": {"dimensions": 768},
  "trace_id": null
}

UniMessage

UniMessage is an object that contains the input for LLMs. Its content items are complete items, typed with a .done suffix.

Example UniMessage:

{
  "role": "user | assistant",
  "content_items": [
    {"type": "text.done", "text": "How are you doing?"},
    {"type": "image_url.done", "image_url": "https://example.com/image.jpg"},
    {"type": "inline_data.done", "mime_type": "image/jpeg", "data": "base64-encoded-image"},
    {"type": "thinking.done", "thinking": "I am thinking.", "fidelity": {"signature": "0x123456"}},
    {"type": "inline_thinking.done", "mime_type": "image/jpeg", "data": "base64-encoded-image"},
    {"type": "tool_call.done", "name": "math", "arguments": {"expression": "2 + 3"}, "tool_call_id": "123"},
    {"type": "tool_result.done", "text": "2 + 3 = 5", "images": [], "tool_call_id": "123"}
  ]
}

Messages saved before 0.5.0 use item types without the .done suffix. They are still accepted, and converted with a deprecation warning, until 0.6.0; normalize_legacy_messages / normalizeLegacyMessages converts stored data.

UniEvent

UniEvent is an object that contains streaming output of LLMs. A stream is a run of delta events, each carrying exactly one content item, closed by exactly one stop event that carries no items but always the usage and the finish reason. Each item streams as one or more .delta fragments followed by its complete .done item, and items never interleave.

Example UniEvents for a tool call:

{"role": "assistant", "event_type": "delta", "content_items": [{"type": "tool_call.delta", "name": "math", "arguments": "{\"expression\": ", "tool_call_id": "123"}], "usage_metadata": null, "finish_reason": null, "created_at": 1694502400000}
{"role": "assistant", "event_type": "delta", "content_items": [{"type": "tool_call.delta", "name": "", "arguments": "\"2 + 3\"}", "tool_call_id": ""}], "usage_metadata": null, "finish_reason": null, "created_at": 1694502400010}
{"role": "assistant", "event_type": "delta", "content_items": [{"type": "tool_call.done", "name": "math", "arguments": {"expression": "2 + 3"}, "tool_call_id": "123"}], "usage_metadata": null, "finish_reason": null, "created_at": 1694502400010}
{"role": "assistant", "event_type": "stop", "content_items": [], "usage_metadata": {"cached_tokens": null, "prompt_tokens": 10, "thoughts_tokens": null, "response_tokens": 12}, "finish_reason": "tool_call", "created_at": 1694502400020}

Read complete items, such as tool calls, from the .done items, and the usage from the stop event.

Token Usage

MMSP provides detailed token usage information through the usage_metadata field of the stop event, the last event of every stream.

The usage_metadata object contains four fields:

• cached_tokens: Cached input tokens
• prompt_tokens: Non-cached input tokens
• thoughts_tokens: Chain-of-thought output tokens
• response_tokens: Non-chain-of-thought output tokens

You can calculate the total token usage as follows:

• input_tokens = cached_tokens + prompt_tokens
• output_tokens = thoughts_tokens + response_tokens
• total_tokens = input_tokens + output_tokens

█████████████  ░░░░░░░░░░░░░ → LLM → ███████████████  ░░░░░░░░░░░░░░░
cached_tokens  prompt_tokens         thoughts_tokens  response_tokens
        input_tokens                          output_tokens

Tracing LLM Executions

We provide a tracer to help you monitor and debug your LLM executions. You can enable tracing by setting the trace_id parameter to a unique identifier in the config object.

async for event in client.streaming_response_stateful(
    message={
        "role": "user",
        "content_items": [{"type": "text.done", "text": "Say 'Hello, World!'"}]
    },
    config={"trace_id": "unique-trace-id"}
):
    print(event)

cd src_py && uv run python -m mmsp.integration.tracer --host 127.0.0.1 --port 25750

cd src_ts && npm run tracer

Then you can view the tracing output in the dashboard at http://localhost:25750/.

LLM Playground

We provide a LLM playground to help you test your LLMs.

cd src_py && uv run python -m mmsp.integration.playground --host 127.0.0.1 --port 25751

cd src_ts && npm run playground

You can access the playground at http://localhost:25751/.
The integrated tracer is available at http://localhost:25751/tracer/.

Wire Protocols

Every client speaks one vendor protocol on the wire, whichever client_type reaches it:

Related Work

• OpenRouter
• Open Responses

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

Used By

Projects built on MMSP:

• PenguinHarness