infai

██╗███╗   ██╗███████╗ █████╗ ██╗
██║████╗  ██║██╔════╝██╔══██╗██║
██║██╔██╗ ██║█████╗  ███████║██║
██║██║╚██╗██║██╔══╝  ██╔══██║██║
██║██║ ╚████║██║     ██║  ██║██║
╚═╝╚═╝  ╚═══╝╚═╝     ╚═╝  ╚═╝╚═╝

Two terminal tools for local-first AI, from one repository.

Each is a single Go binary for macOS and Linux, released and installed separately.
Use either one on its own, or both together.

• infai: local inference
• infaiw: agent harness
• Building from source
• Contributing

infai: local inference

Every inference engine has its own dialect of flags. infai gives them one shape, the
profile: model + engine + config. Define it once, then launch it with a keypress.

Install

brew install dipankardas011/tap/infai          # macOS and Linux

• Linux packages: install infai from the
  openSUSE Software portal,
  which has repository setup for Debian, Ubuntu, Fedora, and openSUSE.
• Script (Linux): installs to /usr/local/bin.

  curl -sL https://raw.githubusercontent.com/dipankardas011/infai/main/install.sh | bash
  

• Binary: download one from Releases
  (Linux and macOS, amd64 and arm64).

Quick start

infai            # open the TUI
infai --version

On first launch infai creates its local database. In the Engines tab, add your
llama.cpp or vLLM binary. In the Models tab, add a model folder and press s to
sync it, or press w to download from Hugging Face. Then create a profile in
Profiles and press enter to launch it.

Features

• Profiles: named launch configurations (context size, GPU layers, KV-cache types,
  flash attention, sampling, ports, extra flags), with engine-aware options for
  llama.cpp and vLLM.
• Runs dashboard: run several models at once, with status, uptime, and live
  tokens per second for each one. Ports are checked before launch.
• Live telemetry and logs: system and per-model CPU, RAM, and VRAM, plus a
  scrollable server log you can copy.
• Model library: scans folders for GGUF and SafeTensors files, reads their
  metadata, and links multimodal projectors automatically.
• Hugging Face downloads: search the Hub, pick a GGUF quantization, and download
  with resumable, atomic transfers. Provenance stays attached to the model.
• Hardware-fit checks: estimates whether a model fits your hardware and suggests
  launch settings before you start.
• 11 themes: tokyonight, everforest, onedark, rosepine, gruvbox, catppuccin, nord,
  dracula, kanagawa, solarized, monokai.

Keys

tab / shift+tab switch tabs, t picks a theme, ? shows help, q quits.

Data

Profiles, folders, engines, and settings live in one SQLite file. Back it up with cp.

Schema migrations run automatically on startup.

infaiw: agent harness

infaiw runs agents as a background service, so a session keeps working after you
close the terminal. The infaiw TUI is a client: attach from several terminals,
leave, and come back to the same session.

• Sidecar loops: delegated subagents that are not done until an acceptance check
  passes. Each loop is its own session and is recorded on the timeline.
• Durable, branching timelines: every session is an event timeline that is
  compact to store and fast to replay, and you can branch from any point.
• Human first: approvals wait for you even after you close the window. Approve,
  deny, or deny with a reason.
• Inter-session messaging: sessions have mailboxes, and the engine routes messages
  between them.
• Task checklists with intent: each task records what was expected, so the agent
  can recover the goal mid-task.
• Hybrid auto-compaction: recent events stay verbatim and older ones are
  compressed.
• Tools: files, bash, web fetch and search, images in and out, skills, MCP servers
  (with OAuth, elicitation, and prompts), and evals.
• Providers: DeepSeek and OpenAI Codex, with configurable thinking levels.

Install and start

The service is opt-in: installing the package does not start it. Run these as your
normal user, without sudo.

Homebrew (macOS and Linux)

brew install dipankardas011/tap/infaiw
brew services start infaiw

Linux packages: install infaiw from the
openSUSE Software portal, then:

systemctl --user daemon-reload
systemctl --user enable --now infaiw.service

Quick start

infaiw provider login    # sign in to a model provider
infaiw                   # open the TUI and start a session

The TUI attaches to localhost:6000 by default. Use --host and --port to reach
another server, and --session <uuid> to resume a saved session.

Keys

Running the service

Homebrew manages launchd on macOS and systemd on Linux. Use
brew services stop infaiw or brew services restart infaiw. Logs are in
$(brew --prefix)/var/log/infaiw.log. Upgrading the formula stops a running
infaiw server, and the service starts the new binary; an inactive service stays
inactive.

If you installed the old cask, migrate once:

brew uninstall --cask infaiw
brew update
brew install dipankardas011/tap/infaiw
brew services start infaiw

The executable moves from the Caskroom to the Cellar. Your user data is not moved
or removed.

• --user runs the service as your account, not root.
• enable starts it on future logins; --now also starts it immediately.
• Follow the logs with journalctl --user -u infaiw.service -f (ctrl+c stops
  following, not the server).
• Package upgrades restart running instances; inactive ones stay inactive.
• Stop it and disable startup with systemctl --user disable --now infaiw.service.
  Your data and configuration are kept.

Without lingering, a user service stops after your last session ends. To keep it
running after logout and start it at boot (may need admin rights):

sudo loginctl enable-linger "$USER"

Packaging follows the Homebrew service DSL,
openSUSE unit guidelines,
and Debian's dh_installsystemduser
with --no-enable.

Configuration

infaiw server is configured only through INFAI_AGENT_* environment variables, for
example INFAI_AGENT_PORT=6000. Background services do not inherit your shell
environment, so put them in ~/.config/infaiw/server.env (chmod 600 if it holds
secrets) and restart the service.

Building from source

Requires Go 1.26+. infai also needs a C compiler, because it uses SQLite through CGO;
infaiw does not.

git clone https://github.com/dipankardas011/infai.git
cd infai
CGO_ENABLED=1 go build -o infai ./cmd/inference
CGO_ENABLED=0 go build -o infaiw ./cmd/agent
go test ./...

Contributing

Bug reports and pull requests are welcome on
GitHub. See AGENTS.md for the
architecture and conventions.

The website lives on the gh-pages branch; edit it by committing there. Provider
catalogs are the exception: they live in catalog/ on main, next to the code that
reads them.

infaiw is being built in public, one post a day; follow along from the
build log.

License

Apache 2.0