Fieldnote

A local-first personal AI assistant that keeps your data in SQLite and uses Algolia for semantic retrieval.

[!IMPORTANT]
This is a personal demo project built for a conference talk. It is not an official Algolia product, is not affiliated with or endorsed by Algolia, Twilio, Sendblue, or Granola, and carries no SLA or support guarantee. It is provided "as is". Read SECURITY.md before exposing an instance to the internet.

What it does

Fieldnote is a personal assistant you actually own. It tracks todos, memories, conversations, and reminders, and can reach you over SMS. Everything lives in a SQLite file on disk that you control.

The interesting part is the split of responsibilities. SQLite is the source of truth for all state. Algolia stores searchable projections of that data, optionally with hybrid semantic retrieval through NeuralSearch, so asking "what did I decide about the pricing work?" finds the right note without exact keyword matches. Agent Studio decides when to search and when to call the app's typed action tools, so the assistant can create a todo or send a reminder rather than only answering questions.

Because Algolia holds only derived projections, reindexing is always safe to repeat and losing the index never loses your data.

Features

• Todos, memories, conversations, and reminders backed by local SQLite, including repeating todos ("every day at 8", "Mon/Wed/Fri at 9pm") that text you ahead of time and roll forward on their own
• Full-text search over your own notes via Algolia, with an optional NeuralSearch toggle for hybrid semantic retrieval (off by default, since it is a paid add-on)
• An agent that calls typed action tools, executed server-side
• Two-way messaging through Twilio SMS or Sendblue iMessage, switchable from Settings, with quiet hours, daily digests, an optional evening check-in that turns your answer into the day's journal entry, and retry with backoff
• Granola meeting-note polling with a manual review queue
• Read-only Jira and Confluence lookups, plus digest briefs: an instruction of your own that the agent runs on its own schedule and texts you, previewable from Settings before it ever sends
• Runs without Algolia credentials in local mode: all CRUD works and conversation search falls back to SQLite, though the agent and NeuralSearch need credentials
• Password-protected sign-in with sessions, so a self-hosted instance is not wide open

Architecture

The whole app is a single Node process. One Express server serves the built React bundle, the JSON API, the message provider webhooks, and an in-process worker that fires every minute for reminders and digests. There is no separate frontend host, database service, or cron service.

Browser  ─┐
          ├──▶ Express (single process) ──▶ SQLite  (source of truth)
Twilio or │         │                            │
Sendblue ─┘         │                            │
                    │                            └──▶ index jobs
                    └──▶ Algolia NeuralSearch ◀────────────┘
                         + Agent Studio

docs/ARCHITECTURE_AND_DATA_FLOW.md has the detail.

Getting started

Requires Node.js 22 (see .nvmrc).

cp .env.example .env
npm install
npm run dev

Open http://localhost:4173. The API runs on port 4174.

Press ⌘K (Ctrl+K) for universal search across memories, todos, and conversation history, and ⌘I (Ctrl+I) to open the agent panel from any page.

⌘⇧D (Ctrl+Shift+D) toggles demo mode, which masks the values that identify one installation — phone numbers, provider account identifiers, the public webhook URL, the Atlassian account, and the agent id — so the app can be screen-shared or recorded as it is. It is a browser-local display setting, also available under Settings > Appearance. Masked inputs keep their real value, so saving a card while it is on writes the stored value rather than the mask.

Without Algolia credentials the app still runs: todos, memories, reminders, and conversation history all work against SQLite, and universal search, memory search, and conversation search each fall back to a SQLite LIKE scan. The agent panel serves a small deterministic fallback assistant rather than Agent Studio. To enable semantic search and the real agent, fill in the Algolia values in .env and run:

npm run seed
npm run setup:algolia
npm run reindex

The server reads ALGOLIA_APPLICATION_ID, ALGOLIA_ADMIN_API_KEY (writes and index settings), ALGOLIA_SEARCH_API_KEY (reads), ALGOLIA_AGENT_ID, and ALGOLIA_AGENT_API_KEY (falls back to the search key if unset). The browser bundle needs its own VITE_ALGOLIA_* copies, which are baked in at build time. .env.example lists every variable with notes.

Then configure and publish the Agent Studio agent using the artifacts in agent-studio/, following docs/AGENT_STUDIO_SETUP.md.

SMS and integrations

The Settings page (/settings) configures encrypted Twilio and Sendblue credentials, which of the two carries outbound messages, reminder delivery, daily digests, quiet hours, Granola meeting-note polling, and Atlassian credentials. Both providers can stay connected, and inbound messages are answered on whichever one they arrive on. Over Sendblue the assistant can also sit in an iMessage group chat you are in: trusted contacts you name in Settings (or everyone in the group, if you say so) can talk to it there, and a reminder for anything asked for in the group is texted back to the group. When someone in a group says who they are (or you say who they are), the assistant saves them as a trusted contact and keeps a "Who's in" memory for the group; something it is asked to say on a schedule, like a daily happy birthday, it writes itself rather than sending as a reminder. It can also read back what was said in a conversation on a given day. Each group gets its own life area, named by the assistant on the first message and editable in Settings, and everything said in that group is filed under it and read only from it, so a question asked in the group never surfaces your own todos and memories. Under Settings → Group chats each group can also get a scheduled morning note about what it has in progress or due soon and an evening question about how the day went, whose answers become one shared journal entry that keeps each person's mood separately (the dashboard's mood chart shows your own by default, with a Shared view for the groups'); you can reword either ask (or have the assistant draft it from what the chat is for) and have a copy texted to your own number. Each group also has its own voice there: the assistant can be told to stay out until someone names it, answers to whatever nickname the group gives it, and keeps a Soul — a few rules on how to talk in that chat, rewritten as the group gives it feedback. You have one too for your own chats, under Settings → Soul. Every turn carries the memories that bear on the message, plus any tagged as preferences, so it does not have to be asked to look you up. Your own turns also carry a short profile of your life, rewritten overnight from your memories, and each group chat has its own profile of the people in it, built the same way from that group's memories and members. It acts like an assistant you delegate to: it looks things up and makes ordinary changes without asking, and checks before anything irreversible. When something you said you'd do is still open the next morning, it texts you once to ask how it went. Inbound handling is entirely server-side — the worker runs the agent loop and executes its tools in-process — so no browser needs to be open for a reminder to send or a text to get answered.

Over iMessage the assistant can also send pictures: a demo-only shopping pair, search_store_products and send_product_cards, searches a small Walgreens-styled over-the-counter catalog (its own Algolia index, seeded from server/catalog/walgreens-products.json) and texts each pick as an image card with a price and a store link. It browses; it never buys. With a GIPHY_API_KEY it can send a reaction GIF, and it can pass on a picture from a page it read. With an OPENAI_API_KEY, pictures people text it are described so it can answer them; Agent Studio itself accepts only text. Every picture texted in is also kept as a file beside the database (data/attachments/, or ATTACHMENTS_DIR), whether or not a key is set, and the Pictures page (/attachments) lists them. A receipt or invoice is read a second time at full detail, so its vendor, date, and amounts are in the description, and a memory the assistant saves from the same message carries the picture, but only a memory in the same chat's scope (a group's pictures never go with the owner's private memories, or the reverse). ATTACHMENTS_MAX_MB caps the total kept, and ATTACHMENT_TEXT_INDEX=off keeps what a picture says out of the hosted search index. Deleting a picture removes the file and also blanks what the assistant read off it in the saved conversation.

With a Bright Data token (BRIGHTDATA_API_TOKEN), the assistant can also search the web and read the pages a search returned, for public facts such as opening hours or news, on every channel including group chats. It runs server-side with a daily lookup cap, and the assistant can only read pages its own search returned or links someone in the conversation sent; see Web.

See docs/SMS_AND_EVENTS.md for tunnels, provider consent requirements, and recovery.

Deployment

The entire app deploys as one container plus one persistent volume for SQLite. A Dockerfile and railway.json are included.

SQLite forces three rules: exactly one replica, a real persistent volume, and no scale-to-zero. That makes serverless platforms unsuitable — notably Cloudflare Containers, whose disk is ephemeral and would silently discard your database on every restart. docs/DEPLOYMENT.md covers Railway, any Docker host, Cloudflare Tunnel for a stable public URL, and why the other options fail.

Commands

Documentation

• Architecture and data flow
• Agent Studio setup
• Tool endpoint mapping
• SMS, reminders, and event integrations
• Deployment
• Troubleshooting and security

Contributing

Contributions are welcome. See CONTRIBUTING.md for the development setup and checks, and CODE_OF_CONDUCT.md for community expectations. Report vulnerabilities privately as described in SECURITY.md.

Author

• Natan Yagudayev (@NatanTechofNY)

License

MIT

No existing application, database, phone integration, or production account is used by this project.