basalt
Health Uyari
- License — License: MIT
- Description — Repository has a description
- Active repo — Last push 0 days ago
- Low visibility — Only 5 GitHub stars
Code Gecti
- Code scan — Scanned 12 files during light audit, no dangerous patterns found
Permissions Gecti
- Permissions — No dangerous permissions requested
Bu listing icin henuz AI raporu yok.
Turn your Mac into a local LLM server: MLX on Metal, OpenAI + Anthropic API + MCP, pure Swift. Point Claude Code at it.
basalt — your Mac is the AI lab now.
Serve, train, quantize, compose and dissect open LLMs, neural speech, video, OCR and image models on Apple Silicon. One Swift 6.4 binary speaks the OpenAI API, the Anthropic API and MCP, and a native macOS 27 Liquid Glass app sits on top. No Python at runtime. No cloud unless you ask for it.
English · Русский · 中文 · Español · Français · 日本語 · Deutsch · Website — bas-alt.vercel.app
Runs open-weight models on Apple Silicon. Logit-verified here: Gemma 4, Qwen3.8 and LFM2.5 (the table below); measured and packaged in huggingface.co/goodmai: Ornith, Bonsai, Qwen3.6-35B-A3B, DeepSeek-OCR-2, OpenJev and Laya (see the models table); gpt-oss and Llama are MLX-compatible. Logos are trademarks of their owners, shown only to indicate compatibility. Anthropic's models are closed-weight, so they are not in the hive; the Anthropic API is supported so that Claude Code can talk to a local model.
Five things in one repo: a local inference server your tools already know how to talk to, LoRA distillation and post-quantization tuning, a docker-compose for multi-model systems, a live X-ray microscope for what happens inside the model, and a native Mac app that puts it all one keystroke away.
Checked against reference logits on every change to the inference path: GoldenLogitsTests replays recorded prompts and fails when a greedy token or a logit moves. Per-family reference logits for qwen3, qwen3_5, gemma4, lfm2 and nemotron_h, dense and MoE, are in Fixtures/ReferenceLogits.
| Golden-tested model | Class | Fixture |
|---|---|---|
mlx-community/gemma-4-26B-A4B-it-qat-4bit |
sliding-window + full attention, MoE | golden |
lukaskremla/Qwen3.8-27B-3bit-MLX-TextOnly |
hybrid linear/full attention, dense | golden |
LiquidAI/LFM2.5-24B-A2B-MLX-4bit |
hybrid conv + attention, MoE | golden |
One-click launch from huggingface.co/goodmai (55+ verified models)
Every model card under goodmai carries a Run with Basalt button (goodmai-run-with-basalt.static.hf.space) that opens Basalt.app via basalt://run?model=... or prints the exact basalt serve / basalt compose CLI command:
| Model / Stack | Role & footprint on 24 GB Mac | Launch |
|---|---|---|
goodmai/gemma-4-12B-it-qat-4bit |
12B QAT daily-driver chat & vision (~7.6 GB, Ladder 4/5 + Lunar 6/6) | |
goodmai/gemma-4-26B-A4B-it-qat-4bit |
26B MoE (A4B), golden-verified (14.9 GB, 56–63 tok/s, Ladder 4/5) |
|
goodmai/gemma-4-31B-it-PV-MACaQ-3.2bpw-MLX |
31B dense PV-MACaQ 3.2 bpw for 24 GB Mac (15.1 GB, 14.1 tok/s, Ladder 4/5 + Lunar 4/6) | |
goodmai/Qwen3.8-27B-PV-MACaQ-4bit-Affine-MLX |
27B dense PV-MACaQ 4-bit (15.3 GB, 11.6 tok/s p50, PPL 1.23, MSE −19.5%, Ladder 4/5) | |
goodmai/Ornith-1.5-35B-A3B-MACaQ-3bit-MLX |
35B MoE (A3B) 3-bit MACaQ (14.6 GB, 64–75 tok/s with #628 switch WHT) |
|
goodmai/Qwen3.8-27B-QTIP-2.5bpw-MLX |
27B QTIP 2.5 bpw trellis (10.0 GB, 16k PPL 4.621) | |
goodmai/Qwen3.8-27B-GSQ-RCO-IQ3_S-Basalt-Native |
27B GSQ-RCO native i-quant (11.8 GB, PPL 4.17, 300/300 System One) | |
goodmai/Qwen3.6-35B-A3B-8bit |
35B MoE in 37.7 GB streamed from SSD at 8.77 tok/s, 0 swap | |
goodmai/Ternary-Bonsai-2-27B-MTPLX-Optimized-Speed |
27B Hadamard ternary in 7.4 GB, 27.1 tok/s with --mtp |
|
goodmai/LFM2.5-24B-A2B-MLX-4bit |
24B hybrid conv + attention MoE, golden-verified (~12.6 GB) | |
Zyphra/ZONOS3-Mini-hf |
44.1 kHz neural TTS across 43 languages (2.24 GB, RTF 0.23–0.26) | |
goodmai/DeepSeek-OCR-2-8bit |
Vision-encoder OCR with official 768 tiling (/v1/pipeline) |
|
goodmai/OpenJev-Flash-9B-MLX-4bit |
9B local System One decider (98.9% acc, −77% LLM tokens) | |
24gb-voice-zonos |
Recommended 24 GB Compose stack (Chat + ZONOS3 + Kokoro ANE) |
Inference: 40B MoE at 67 tok/s, 27B in 7.4 GB, or 37.7 GB MoE with zero swap — behind the APIs you already use
- Claude Code on your own GPU in one line.
ANTHROPIC_BASE_URL=http://localhost:8080 ANTHROPIC_AUTH_TOKEN=local claude— your real API key is never touched. OpenAI SDKs, OpenCode, Continue, Cline and aider connect over the OpenAI API, Claude Desktop and Cursor over MCP (with per-tool and per-group--mcp-enable/--mcp-disable, #488). - Fast on a laptop. On an M4 Pro with 24 GB: Recursive
Flint-40B-A2.5B(oQ3e) decodes at 67.02 tok/s in 17.61 GB (#522);Ornith-1.5-35B-A3B-MACaQ-3bitdecodes at 64–75 tok/s in 14.6 GB with our single-passMACaQSwitchLinearWHT +gatherQuantizedMMkernel (3.41×faster MoE kernel,+82%end-to-end on Gemma 4 26B-A4B, #628);Qwen3.5-9B-PV-MACaQ-4bitreaches43.2 tok/sp50 (+52%speed,−33%RAM vs 8-bit) with exact4/5Ladder parity;Gemma-4-31B-it-PV-MACaQ-3.2bpwfits 31B dense in15.1 GB(14.1 tok/s, Ladder4/5+ Lunar4/6) on 24 GB Mac where 4-bit OOMs; Qwen3-4B decodes at 78–89 tok/s; Ternary Bonsai 2 27B fits in 7.4 GB and runs at 20.9–21.5 tok/s, 27.1 with its MTP head and 31–38 with prompt lookup on code (#239, #263);.ggufGSQ-RCO /i-quantcheckpoints execute directly on custom Metal kernels (#433, #515). - Models larger than RAM, zero swap.
Qwen3.6-35B-A3B-8bit(37.7 GB) decodes at 8.77 tok/s on a 24 GB Mac with 0 swap via batched POSIX AIO andLIO_NOWAITnext-layer SSD expert prefetch (--expert-streaming, #477, #478, #516), alongside UMA-aware L0/L1/L2 tiered KV caching (TieredKVHierarchyCore) and adaptive heterogeneous planning (#627). - Speculative decoding that stays exact. Prompt lookup gives 1.84–2.66× on code edits (Bonsai 2 27B, #239) and 3.45–3.62× (Gemma 4 12B, #265), with greedy output unchanged;
SpecTrOT +Residual Race,EAGLE-3,SmallM(M=2..16),Fast-dLLM+DFlashandDiffuCDextend lossless multi-draft and block-diffusion decoding (#593, #603). - The next turn costs only its new tokens. A prefix-tree prompt cache, even for hybrid models: on Ornith-1.5-9B with a 12.6k-token agent prompt, the second turn takes about 1 s instead of about 36 s (#82).
- Long contexts in less memory. KV quantization with exact zones (
4|8|k8v4, plusKVarN2D balancing, unbiased 1-bitQJL,ChanMix/ThinKV, cross-lingualHeadKVRBO transfer, centroidRQ-KV,turbo3|turbo4|turbo8v4andpolar4RoPE-pair keys, #472, #590, #602): a 16k prompt peaks at 1.03 GB instead of 3.11 GB on Qwen3-4B (#159). - Video, neural TTS, full-duplex speech, music & 2K MLX image synthesis. Phased 24 GB UMA Video Studio runs
LTX-2.5 AV-DiT,Kandinsky 6.0 CrossDiT,LingBot-Video MoEandWan 2.2with causal 3D-VAE, brick-tiled STA Metal kernels andAVAssetWriter.mp4muxing (#608–#611);LFM2.5-AudioMimi/SNAC duplex codecs,SmartTurn+Sortformer,MossFormer2,Soprano-80M,Qwen3ForcedAligner,Zyphra/ZONOS3-Mini-hf(44.1 kHz, 43 languages, 2.24 GB, RTF 0.228–0.263, #527) andKokoro-82Mpower/v1/audio/*(#604);FLUX.34-stage compose,SeedVR2upscaling,ACE-Step 1.5music,Z-Image-Turbo,Iris-3BandSANA-1.5power/v1/images/*and/v1/music/*(#542, #605). - A whole local stack + Swift SDK. Tool calls, JSON-schema, regex and grammar output (XGrammar), Meta
Muse-Glimmer-30B(#534),Ripple-65B,Nemotron-3.5-Lightning(#522),DeepSeek-OCR-2(#476),/v1/embeddings,/v1/rerank,BasaltFoundationModelsSwift SDK (#606),swift-huggingfaceXet downloads (#587), LoRA/PHM/OFT adapters picked per request, and ordered LAN-before-cloud fallback tiers (#480). - PV-MACaQ 3-bit makes 27B fit and fly on 24 GB.
Qwen3.8-27B-PV-MACaQ-3bitpasses the Official Ladder 5/5 (includinglunar6/6) in 11.5–12.4 GB at 15.6–16.7 tok/s on an M4 Pro, against 4/5 rungs, 15.3 GB and 11.6 tok/s for the stock 4-bit.--speculative-lookupnow also runs on Qwen3.5/3.8 checkpoints that ship a vision tower (43–51% draft acceptance, +17% on code edits), and--kv-bits kvarn|kvarn-qjl(2-bit KV) and--attention-backend mobaare wired intobasalt serveand pass a 4k passkey (#632, #634). Checkpoints:goodmai/Qwen3.8-27B-PV-MACaQ-3bit-Affine-MLX,goodmai/Qwen3.8-27B-Heretic-PV-MACaQ-3bit-Affine-MLX.
Details: REST API · integrations · commands and flags · models · benchmarks
Training & Quantization: universal any-to-any converter, PV-MACaQ, distributed & QAT
- Universal any-to-any quantization converter (
basalt models convert). Convert acrossMACaQ(affine/mxfp4/moe2.8 bpw router-weighted Hessian, #537, #599),FlatQuant&LIRQuant(#588, #594),SINQ/A-SINQ/EAQuantdual-axis normalization (#561),QuIP# E8P2-bit lattice &TurboBoABabai–LLL (#589),QTIPbitshift trellis (2.0–3.0 bpw, #494),SVD-LLM V2/OBR(#592),.ggufi-quants &MACaQ-R, plus domain-awareQAT STErefinement and post-quantization quality/fairness/leakage audit (QuantizationAuditCore, #623) — wired identically across CLI, REST/v1/models/convert*and macOSConvertModelSheet(#546). - Post-quantization PV-Tuning & PV-MACaQ.
basalt models convert --to macaq:affine:4:64 --pv-tunealternates continuous scale/bias updates with discrete Hessian flip-refinement over Walsh–Hadamard blocks: onQwen3.5-9B,8-bit → 4-bit PV-MACaQcuts RAM by 33% (9.97 → 6.68 GB) and speeds up decode by+52%(27.3 → 41.6 tok/s, p5043.2) with exact4/5Ladder parity (valKL = 0.0044); on 2–3-bit checkpoints, PV-Tuning cuts perplexity loss by 8.7–29% in under 3 minutes (#486, #545). - Distill, post-train & distribute across Macs.
basalt models distilldistills OpenAI/Anthropic teachers intolora,phm,oftor quantizedAdaLoRA (LR/40)adapters (#501, #508, #624); two-stageSFT → DPO, zero-copyADPO/cADPO,EMA-QLoRA,Self-Appraisal, andRLVRanti-shortcut audit keep preference alignment sharp (#601, #624); multi-Mac cluster training supports authenticated transport,DDP + ZeRO-2,DiLoCo+PowerSGD,Zero-Bubble ZB-H1andExpert-Parallel MoE(#595–#598). - Train while you serve, with a quality gate. Time-sliced MCP
distill, verified-onlytrain_experts,train-jepa, and bit-for-bit rollback if held-out KL or perplexity regresses.
Details: training commands · self-training, distillation and the quality gate
Compose: declarative basalt.yaml v2.2, business governance & 84 hardware profiles
- docker-compose, for models (
basalt.yaml v2.2). Local and cloud models, RAM deduplication planner, specialist profiles,resident/on-demand/sequentialmemory modes,BusinessOrchestrator5-component routing,LANTERNsemantic caching,HITLapproval gate (/v1/mcp/approvals), andLAAS-CNRSneuro-symbolic knowledge core (SymbolicKnowledgeGraph) — in one declarative file (#607, #625, #626). - 84 built-in profiles for every Mac (
16 GBto256 GB). Seven unified-memory tiers (16gb,24gb,32-36gb,48gb,64gb,128gb,256gb) × 12 workload archetypes launch in one command (basalt compose up 24gb-voice-zonos, #528). - Refuses what will not fit, changes live.
basalt compose plansizes models from the Hub before download and refuses overflows;compose diff/compose uphot-reload diffs cleanly;compose benchruns declarative benchmark suites and exports leaderboard bundles (#505). - Local judges, calibrated routing & cloud budgets.
OpenJev-Flash-9B-4bitreaches 98.9% triage accuracy while cutting downstream LLM tokens by 77% (#465), temperature-scaledCalibratedRouterGate+Gemini Decision 1route hard queries cleanly within daily USD caps (#418, #622), and perceived document/audio text never leaves your Mac by default.
Details: compose reference and measurements · pipeline presets and memory modes · cloud decisions with Gemini
Research: open the model up while it runs
- Model X-ray, per-token attribution & Belnap epistemic gating.
model_describe(reportingMACaQ/FlatQuant/LIRQuantmodule manifests, #630),model_tensors,model_tensor,kv_inspect,trace_run,trace_tokens(with automatic Gemma BOS handling, #630),/v1/xray/tokensper-token logprobs & hallucination risk (#613),Score d'influencehead attribution (#622), Marchenko–Pastur spectra, four-valued BelnapSIXTEEN_3evidence states (#500), complex MVN phasor probes (#509), Fisher–Rao drift (#503), and OpenMetrics at/api/v1/metrics(#458). - Reversible surgery & mechanistic circuits. With
serve --research-tools:intervene(including conditional ablation, GQA group patching, cognitive nullspace & anti-steering guard, #600),MAO-SNMFfeature decomposition,weights_delta, livetensor_pack/tensor_unpack,activation_contrast,steer_search/steer_commit,experiment_compare, andintrospect. Every experiment is an undoable overlay with a TTL and journal; weights on disk are never modified. - Benchmarks that run the model's code. A five-rung ladder ending in an Earth-to-Moon mission executes the model's own code against unseen inputs, and ARC-AGI runs under official pass@2 rules.
Details: research tools · research foundations (95+ papers) · benchmarks · serve flags
Basalt for Mac: macOS 27 Liquid Glass, one keystroke or one click away
- A native SwiftUI app on a supervised engine. Built for macOS 27 Liquid Glass (#479) in English and Russian (#456): 257 ms warm window, 0 idle wakeups, 7.4 ms p95 per streamed token in a 2,000-message transcript, and 40 ms
⌥Spacequick-ask overhead (#463). - Compose Video Studio, 2K Image Studio & Universal Converter UI. Generate and play A/V
.mp4videos on 24 GB Macs (#611), edit 2K RGBA images with Before/After slider (#519), convert & QAT-audit models directly inConvertModelSheet(#623), and browse/publish on Hugging Face with 10 Hz SSE telemetry & Xet downloads (#587). - Compose Profiles (
⌥⌘C), full-duplex voice &basalt://1-click launch. Auto-recommends top profiles for your Mac's RAM tier (16–256 GB, #528), streams hands-free voice viaSpeechAnalyzer/LFM2.5-Audio/ZONOS3-Mini(#604), and handlesbasalt://run?model=...links,⌥Spacequick ask, Shortcuts, Spotlight, and notarized.dmgpackaging.
Basalt.app: Compose running on the engine and the chat with its inspector (macOS 27 Liquid Glass, dark & light):


Details: Basalt for Mac
Quick start
Requirements: macOS 27+, Xcode 27.0+ / Swift 6.4 with the Metal toolchain (xcodebuild -downloadComponent MetalToolchain), Apple Silicon M1–M4, 2–38 GB of disk depending on the model. Full steps, building from source and the ./basalt-claude launcher are in docs/install.md.
No Xcode, no build — run the prebuilt Apple Silicon binary in two lines (macOS 27+, Apple Silicon; mlx.metallib ships inside the archive; the first run downloads the 12 GB model, use goodmai/gemma-4-12B-it-qat-4bit for a lighter start):
curl -fsSL https://github.com/goodmai/basalt/releases/latest/download/basalt-macos-arm64.tar.gz | tar -xz
./basalt-macos-arm64/basalt serve --model goodmai/Qwen3.8-27B-PV-MACaQ-3bit-Affine-MLX --speculative-lookup --rest
brew tap goodmai/basalt https://github.com/goodmai/basalt
brew trust goodmai/basalt
brew install goodmai/basalt/basalt # builds CLI & server from source
basalt onboard # picks a model that fits your Mac and downloads it
basalt serve --model goodmai/gemma-4-12B-it-qat-4bit --rest
# in another terminal: Claude Code, backed by your own GPU
ANTHROPIC_BASE_URL=http://localhost:8080 ANTHROPIC_AUTH_TOKEN=local claude
Stream a 37.7 GB MoE model on a 24 GB Mac, or run a full multi-model compose system:
basalt serve --model goodmai/Qwen3.6-35B-A3B-8bit --expert-streaming --rest
basalt compose plan 24gb-voice-zonos
basalt compose up 24gb-voice-zonos # 84 profiles across 16–256 GB Macs
The Mac app, from a checkout (registers basalt:// 1-click links automatically on first open):
./scripts/bundle_app.swift # release build, Metal kernels, then .build/Basalt.app
open .build/Basalt.app
Local 4B–12B models are not a drop-in replacement for a frontier model — they are for private, offline, zero-cost agentic work and for hybrid local/cloud systems. basalt fit tells you what your machine can actually run. The model table lists what broke too.
Star it if you want more MLX-native tooling in Swift, and open an issue with your Mac and your tok/s — PRs adding a row to the benchmark table are welcome.
Documentation
Translated documentation is available under
docs/i18n/: Русский · 中文 · Español · Français · 日本語 · Deutsch.
| Topic | Doc |
|---|---|
Install, requirements, build from source, ./basalt-claude |
docs/install.md |
CLI commands, serve flags, prompt cache, CacheBlend, PV-Tuning |
docs/commands.md |
| REST API: security, model management, OpenAI, Anthropic, images, SSE, OpenMetrics, Swagger | docs/rest-api.md |
| Claude Code, OpenCode, OpenAI SDKs, MCP for Claude Desktop and Cursor | docs/integrations.md |
Verified models, goodmai Hub stack and downloading |
docs/models.md |
| Features at a glance, benchmark ladder, ARC-AGI, provenance | docs/benchmarks.md |
| Pipeline presets and memory modes | docs/pipelines.md |
| Cloud decisions: Gemini Decision 1, multimodal media, the calibration teacher | docs/cloud-decisions.md |
basalt compose reference and measurements |
docs/compose.md |
Basalt for Mac & basalt:// deep links |
docs/mac-app.md |
| Model X-ray, telemetry, experiments, self-training, distillation | docs/research-tools.md |
| Mathematical & scientific research foundations (85+ papers & acknowledgements) | docs/research-foundations.md |
| Project structure, architecture notes, scripts, dependencies | docs/architecture.md |
| Architecture patterns and modules (Russian) | docs/ARCHITECTURE_PATTERNS_AND_MODULES.md |
| Cloud: OpenRouter setup, cost optimization, model comparison | docs/cloud/OPENROUTER_SETUP.md · docs/cloud/COST_OPTIMIZATION.md · docs/cloud/MODEL_COMPARISON.md |
| Working on basalt with AI agents | docs/AI_SETUP.md · docs/AGENT_OPERATIONS.md |
| Roadmap, privacy policy | docs/ROADMAP.md · docs/PRIVACY_POLICY.md |
| Research notes on agent CLIs, checklists, specs (Russian) | docs/features.md · docs/specs/FEATURES_CHECKLIST.md · docs/specs/AUDIT_CHECKLIST.md · docs/specs/EMOJI_AS_CODE.md |
| Contributing, changelog | CONTRIBUTING.md · CHANGELOG.md |
License
MIT © 2026 goodmai
Model weights are not covered by this license — each model on HuggingFace carries its own terms.
Yorumlar (0)
Yorum birakmak icin giris yap.
Yorum birakSonuc bulunamadi