Smart Glasses AI Agent

Your Ray-Ban Metas. Finally smart.

An open-source, hands-free AI agent for Ray-Ban Meta glasses.
Say "Ronie" and just talk. It sees what you see, browses the web for live answers,
runs your TV and laptop, and remembers your life. Free to run. No subscription.

_{Not affiliated with, endorsed by, or sponsored by Meta Platforms or Ray-Ban (EssilorLuxottica). "Ray-Ban Meta" only names the glasses this works with.}

https://github.com/user-attachments/assets/24abf23e-2b7b-436d-a97f-da1470be848b

_{Launch Video · sound on}

Things you didn't know glasses could do.

Just say it: ten things Ronie does. All your screens, no remote needed, chess and poker, interview mode, reads the menu, names the car, remembers for you, the live web, interrupt any time, and Hindi, English or both.

Read all ten

"Explain how black holes form."

You're on the sofa. The answer starts in your ear, a video starts on the TV, and an article opens on the laptop in front of you. It took a photo to see which screen you're facing.

"Turn on the PlayStation."

The PS5 wakes, the TV switches to it, and the volume lands at exactly 15. You never touched a remote.

"What's my best move?"

You glance at a chess board. A chess engine reads it and says the move. At a poker table it gives you the odds.

"Switch to interview mode."

In a real interview, it listens through your laptop and whispers short answer points while you talk. Practise first with a mock interview.

"What's this dish, and is it worth it?"

You look at a menu in another language. It translates it, says what to order, and roughly how many calories you're about to eat.

"Which car is that?"

Make, model, roughly what it costs. Just from a look.

"Remember, I parked on level 3, pillar B12."

Three hours later: "Where did I park?" It knows. It remembers your people, plans and preferences too, across every model you use.

"Who won the race today?"

Ronie has the web at its fingertips. It searches, opens and reads the top pages itself, and gives you one clean sentence, not ten blue links. Scores, news, prices, opening hours: anything that changed today.

"Stop."

It stops talking instantly and listens. Interrupt it any time, like a person.

Hindi, English, and the mix of both.

Talk the way you talk. More languages are on the way.

Compared with the built-in assistant.

	This project	Built-in Meta AI
Controls your Android TV, apps and HDMI devices like a PS5	✅ Yes	❌ No
Opens videos and pages on your laptop, across both displays	✅ Yes	❌ No
Shows things on whichever screen you're facing	✅ Yes	❌ No
Chess moves and poker odds from what you see	✅ Yes	❌ No
Quiet answer points in a real interview, and mock interviews	✅ Yes	❌ No
Long-term memory you can see, edit and delete	✅ Yes	❌ No
Pick your model: Gemini, Claude, ChatGPT, DeepSeek and more	✅ Yes	❌ No
Open source, on your own server, free APIs	✅ Yes	❌ No

Ronie App

A tour of the Ronie Android app: Home with a greeting, a rotating 3D brain showing what Ronie remembers and a live voice wave; Modes; Memory; Devices; and the AI models. Modes: Assistant, Interview mode, Interview practice, Coding, Translator, Game, Focus, Cooking Memory: everything Ronie knows about you, editable Devices: start with your glasses, the glasses camera, TV, laptop and the brain server AI: the live voice model and every provider's status

Ronie's brain turns on Home, lighting up whatever it remembers about you. Pick a mode, see and edit every memory,
and set up your glasses, TV, laptop and AI keys, all from your phone. A guided setup walks new users through it.

Download the app (APK)

How it works.

How it works: the glasses talk to the Android app over Bluetooth; the app streams voice to the AI model and talks to the brain server over WebSocket; the brain runs tools and a model router and reaches the laptop agent; the phone controls the TV over home Wi-Fi.

Six pieces, one conversation. Here's the path a single sentence takes.

1 · The glasses. Ray-Ban Meta Gen 2, unchanged. Their microphones, speakers and camera reach your phone over Bluetooth. Nothing is installed on them.

2 · The Android app. It hears the wake word "Ronie" on the phone itself, then opens a live voice session and streams your voice. It takes photos with the glasses camera through the Meta Wearables SDK, knows when you're home from your Wi-Fi, and holds the TV remote.

3 · The AI model. Your voice goes to Gemini Live, which listens, speaks back and can be interrupted mid-sentence. When a question needs real thinking, the brain can hand it to Claude, ChatGPT, DeepSeek, Groq, OpenRouter or a local Ollama model.

4 · The brain. A small Python server (FastAPI) that you run. It holds the instructions, the tools (web search, vision, chess, memory, show-on-screen), your long-term memory and the model router, which is free by default. The app talks to it over an encrypted WebSocket.

5 · The laptop agent. A tiny Windows program. When the brain says "show this," it opens the page or video, lays windows out across your displays, controls media and volume, and in interview mode hears the call audio.

6 · Your TV. Any Android TV or Google TV. The phone talks to it directly over your home Wi-Fi: it opens apps, plays shows, sets the exact volume, and switches HDMI inputs to wake a PS5.

Brain: It gets to know you.

The brain: what you say over days and months becomes key facts in memory and a growing profile of you, stored on your server and never shared.

One memory, shared by every model and every conversation.

It remembers what matters. Say "remember…" and it's saved. It also picks up the details a friend would: people, places, plans, tastes.
Nothing gets lost. Old conversations fold into a running summary. Facts that change are updated; facts that stop being true are dropped.
Ask it your way. Hindi, English or Hinglish: "meri gaadi kahan khadi hai?" finds the parking spot you saved in English.
The newest is the truth. Park again and the old spot is replaced. Plans and reminders clear themselves once their day has passed.
A picture of you takes shape. Weeks in, it knows your routines. A year in, it knows you.
Every model starts from it. Switch from Gemini to Claude and you never repeat yourself.
You're in control. See, edit or delete any memory from the app.

Private by design.

Security and privacy: traffic is encrypted; only paired devices get in; memory, keys, settings and the admin page stay locked on your own server; only the question goes to the AI provider you choose.

There's no company in the middle. You run the brain, so your data lives where you put it.

Your server. A free Oracle Cloud machine in your account, or your own computer at home.
Encrypted. HTTPS everywhere, with a free Let's Encrypt certificate.
Paired devices only. Each device gets a private token. Everything else is refused.
Your data stays put. Memory, profile, settings and API keys live on your server. Keys never go back to the app.
No public admin page. Mission control is reached only through a private SSH tunnel.
Only the question leaves. It goes, with the memory it needs, to the AI provider you picked, on your own key.
On-device where possible. The wake word runs on the phone. The "which screen" photo is taken only at home, used once, never stored.
Open source. No analytics, no ads, no tracking. Read every line.

Get started.

You need Ray-Ban Meta glasses, an Android 14+ phone, and a free Gemini API key. A Windows laptop and an Android / Google TV are optional.

Get free API keys: Gemini (required), Groq and OpenRouter (recommended).
Run the brain on your laptop, or on a free cloud server so it works away from home.
Install the Android app: download the APK (Android 14+), or build it from android-app/. Its setup scans your brain's pairing QR code.
Optional: pair your TV in the app, and add your Windows laptop with one PowerShell command:
irm https://raw.githubusercontent.com/raunakpatil/smart-glasses-ai-agent/main/install-agent.ps1 | iex

The full guide is in docs/SETUP.md.

Cost: nothing. Gemini, Groq and OpenRouter free tiers, plus Oracle Cloud's Always Free server if you want it online. Claude, ChatGPT and DeepSeek are optional and only used if you add a key.

Coming Soon.

Coming next, in development: it makes calls for you, says awkward things for you, books the trip, names the song, watches for your bus, and remembers where you left your keys.

Contributing

Issues and pull requests are welcome. Run the brain tests with pytest in brain/.

Fonts: Inter Tight and JetBrains Mono, under the SIL Open Font License (see android-app/fonts-license/). MIT licensed, see LICENSE.