locallm

mcp
Security Audit
Fail
Health Warn
  • License — License: Apache-2.0
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Low visibility — Only 8 GitHub stars
Code Fail
  • rm -rf — Recursive force deletion command in examples/aiql/packaging/build-and-sign.sh
  • Hardcoded secret — Potential hardcoded credential in examples/api-lab/chat_app/app.py
  • Hardcoded secret — Potential hardcoded credential in examples/api-lab/test_conformance.py
  • Hardcoded secret — Potential hardcoded credential in examples/api-lab/test_error_handling.py
Permissions Pass
  • Permissions — No dangerous permissions requested

No AI report is available for this listing yet.

SUMMARY

Free Swift SDK for adding on-device AI, tools and MCP servers to a Mac app. Apple’s on-device model, open-weight models (MLX) and hosted models behind one API, with a security gate on every tool call.

README.md

LocalLM SDK

Add a chat assistant that can use tools to your own Mac app, with the model running on the Mac: Apple's
on-device model or an open-weight model, or a hosted model when you want one. No per-token bill for local models,
and no data leaving the machine unless you choose. The SDK is free. Release notes are in the CHANGELOG,
and the project site is thisbrain.ai/locallm.

A new name. From 3.0 this is the LocalLM SDK; it was the LocalLM Lab SDK. LocalLM Lab is the Mac app built on it.
Nothing in your code changes: the modules, products and release files keep their names (LocalLMLabSDKCore,
LocalLMLabSDKInference, …), and so does the API (LocalLMLab, lab.makeSession).

Looking for the LocalLM Lab app (a Mac app for trying models, tools, MCP servers and prompts before you build with
them)? That is a separate page. This page is for developers.

What it looks like

A model, a tool and a gate that asks the user, in a SwiftUI app. Add the SDK as a Swift package binary target
(the quickstart has the manifest), then:

import LocalLMLabSDKCore
import LocalLMLabSDKComponents

extension RouteName { static let chat: RouteName = "chat" }     // a route is a name for "the model for this job"

let lab = LocalLMLab(configuration: .init(providers: [SystemModelProvider()]))
lab.models.route(.chat, to: .system)                           // swap the model here, nowhere else

let confirmations = ToolConfirmationPresenter()                // the approval sheet, from Components
let session = try lab.makeSession(
    route: .chat,
    tools: [ClockTool()],                                      // plus your own tools, or an MCP server's
    authorizer: ConfirmingToolAuthorizer(channel: confirmations,
                                         hostTools: .ask(atOrAbove: .mutate, by: .user)))

for try await _ in session.streamResponse(to: "What time is it?") {}
// session.hostTranscript is @Observable: your SwiftUI view reads it and redraws as the reply streams.
// Put .toolConfirmationSheet(confirmations) on your root view, and a tool that changes data asks first.

The model never runs code: it writes a tool call, your app decides whether to run it, and the result goes back. The full
chat, with tool-call rows, servers and saved conversations, is examples/mcp-chat.

Start here

  1. Quickstart: a working SwiftUI chat in about 50 lines. Needs macOS 27 with
    Apple Intelligence on, and Xcode 27 to build.
  2. Examples: complete apps, from a command-line tool to a full chat with servers and
    tool-call rows. examples/mcp-chat is the one to read for a chat UI,
    and answersearch-oss shows hosted web search in a menu-bar app.
  3. The SDK guide: the full reference.

What is in the SDK

Framework What it gives you Form
LocalLMLabSDKCore The model layer (routes, sessions, Apple's on-device model), tools, an MCP client, the security runtime, connectors Binary
LocalLMLabSDKInference Open-weight models on the Mac through MLX, with a prompt cache Binary
LocalLMLabSDKRemote Hosted models: GPT, Claude's online API, OpenRouter, any OpenAI-compatible server Binary
LocalLMLabSDKClaude Claude through Apple's Foundation Models interface Binary
LocalLMLabSDKComponents Prebuilt SwiftUI: chat views, server and model settings Source in Components/
LocalLMLabSDKMCPAppsHost A server's own interactive views shown in a chat Source in MCPAppsHost/

MLX models, hosted models and Private Cloud Compute need macOS 27; Apple's on-device model runs on macOS 26. The
SDK is proven under App Sandbox, with a signed path to both Developer ID distribution and the Mac App Store.
LocalLM Lab itself runs on it.

New in 3.0

  • A chat that stays fast with many tools. Tools are available, not attached: a small picker chooses the few
    each message needs, and a server is contacted only when one of its tools is used. See
    MCP in your app.
  • A prompt cache for local models, on by default, so follow-ups and tool loops don't re-read the whole
    conversation. See the prompt cache.
  • Every tool call passes an authorizer. The authorizer is the code that decides whether a call may run. A session
    built without one now refuses anything that could change data; in 2.3 it ran every call unasked. See
    the tool authorizer (it has the upgrade steps).
  • Safe by default with tools you didn't write. Around the authorizer: a server that changes its tools is held for
    review, outside content is treated as data, programs run in a sandbox, and decisions go to an audit stream. Start with
    the security overview, then the guide for host apps.
  • Loading a model reads only the disk. Downloading is a step your app takes.
  • More of MCP: resources and prompts in chat, richer tool results, paste-a-setup.

Upgrading? 3.0 is a breaking release: read docs/migrating-to-3.0.md. From 1.x, read
migrating-to-2.0 first.

Build with an AI coding agent

The repo is set up for Claude Code, Codex and other coding agents (setup and prompts on the site). AGENTS.md lists what an agent can't
guess (confirm names in docs/api-surface.md, give every session an authorizer:, loading never downloads), and the
locallm-swift-app skill carries the rest.

  • Work inside a clone. git clone this repo and start your agent in it. Claude Code picks the skill up from
    .claude/skills/; other agents read AGENTS.md, which points to the skill's files.
  • Work in your own project. Copy skills/locallm-swift-app into ~/.claude/skills/ (Claude Code), and give the agent the
    docs: clone this repo beside your project and start with claude --add-dir ../locallm, or rely on the skill's fallback to
    the same files on the 3.0.0-GA branch.

A prompt that works well says what the agent can't read from the SDK: the version, which models, how the app is
distributed, who approves tool calls, and how to check the result.

Use the locallm-swift-app skill. In this Xcode project, add a SwiftUI chat on LocalLM SDK 3.0.0-GA using Apple's
on-device model and the Calendar tool. It is a direct-download app with a window, so ask the user before any tool that
changes data. Start from examples/mcp-chat for the chat window and examples/plate-today-tools for the Calendar tools, and confirm API names in docs/api-surface.md, not from memory. When you
are done, run a turn that creates a calendar event and show me it was not denied.

Use the locallm-swift-app skill. Add Qwen3-4B-4bit through MLX to this app. Loading must not download silently: check
the size, ask the user, then download. Show a progress bar, and handle a model that is not on disk.

Documentation

About this repo and support

ancientcomputing/locallm is the public home of the SDK's releases, documentation and examples. The SDK's source
is closed; Components and MCPAppsHost ship as source. The 3.0.0-GA release is on the
3.0.0-GA branch.

The SDK is free. ThisBrainAI also offers support agreements for teams that build on it, including security review
questions: see thisbrain.ai/locallm. To suggest a feature, raise an issue.

Reviews (0)

No results found