The Vision

Traditional assistants wait for a command. B waits for a moment.

B is designed as a proactive desktop companion. Using a 60fps event-driven central nervous system, B synchronizes his emotional state with your environment. He sees your screen semantically, tracks your active focus, and intervenes only when he has something truly valuable to contribute.

• Silent by Default : B respects your deep work state.
• Emotionally Aware : A complex internal state machine drives expressions and curiosity.
• Agentic Autonomy : B doesn't just respond; he thinks, wonders, and observes.

Architecture Overview

B is built on a centralized asynchronous pub/sub event bus : the EventBus. Every module communicates exclusively through this bus. No module knows about any other module. This strict decoupling makes the system testable, maintainable, and resilient.

flowchart TB
    subgraph P["Perception Layer"]
        WT["WindowTracker"]
        SS["SemanticSensor"]
        VS["VisionSensor OCR"]
    end

    subgraph C["Cognitive Layer"]
        CE["CognitiveEngine"]
        SM["StateMachine"]
        AE["AutonomyEngine"]
    end

    subgraph O["Output Layer"]
        VE["VoiceEngine"]
        FR["FaceRenderer"]
        CB["ChatBubble"]
        KE["KinematicsEngine"]
    end

    subgraph I["Core Infrastructure"]
        EB["EventBus"]
    end

    WT -- "active_window_changed" --> EB
    SS -- "context_updated" --> EB
    VS -- "context_updated" --> EB
    EB -- "context_updated" --> CE
    EB -- "tick" --> SM
    EB -- "tick" --> KE
    CE -- "b_spoke" --> EB
    EB -- "b_spoke" --> VE
    EB -- "b_spoke" --> FR
    EB -- "b_spoke" --> CB
    CE -- "b_move_request" --> EB
    EB -- "b_move_request" --> KE
    AE -- "trigger_proactive_thought" --> EB
    EB -- "trigger_proactive_thought" --> CE

System Workflow

The data flows through B in a deterministic pipeline: Perception → Cognition → Expression.

sequenceDiagram
    participant WT as WindowTracker
    participant SS as SemanticSensor
    participant VS as VisionSensor
    participant CE as CognitiveEngine
    participant VE as VoiceEngine
    participant FR as FaceRenderer

    WT->>SS: active_window_changed
    activate SS
    SS->>SS: UIA Tree Walk
    alt Extraction Success
        SS->>CE: context_updated
    else Failure / Cooldown
        SS->>VS: semantic_extraction_failed
        VS->>VS: OCR Capture
        VS->>CE: context_updated
    end
    deactivate SS

    CE->>CE: LLM Inference
    CE->>VE: b_spoke (Text + Emotion)
    activate VE
    VE->>VE: TTS + DSP Vocoder
    VE->>FR: speaking_start
    VE->>VE: Audio Output
    VE->>FR: speaking_end
    deactivate VE
    CE->>FR: b_move_request (Spatial)

Internal Module Structure

Core Infrastructure

Perception Layer (Sensors)

Cognitive Layer (The Brain)

Output Layer (Expression)

Data Flow

flowchart LR
    subgraph Input["Input"]
        A["User Types"]
        B["User Speaks"]
        C["Screen Changes"]
    end

    subgraph Process["Processing"]
        D["EventBus"]
        E["CognitiveEngine"]
        F["StateMachine"]
    end

    subgraph Output["Output"]
        G["FaceRenderer"]
        H["VoiceEngine"]
        I["ChatBubble"]
        J["Kinematics"]
    end

    A -- types --> D
    B -- speaks --> D
    C -- changes --> D
    D -- routes --> E
    E -- drives --> F
    F -- animates --> G
    F -- speaks --> H
    F -- displays --> I
    E -- moves --> J

Request Lifecycle

stateDiagram-v2
    [*] --> Idle
    Idle --> Listening : user_spoke / voice detected
    Listening --> Thinking : 1.2s delay
    Thinking --> Streaming : first token received
    Streaming --> Speaking : sentence_ready
    Speaking --> Streaming : next sentence
    Streaming --> Idle : [SILENCE] / end of response
    Speaking --> Idle : finished_speaking + linger
    Idle --> Proactive : autonomy trigger
    Proactive --> Thinking : context available
    Proactive --> Idle : no context / silence

Project Structure

B/
├── main.py                  # Entry point : boot sequence
├── core/
│   └── bus.py               # EventBus : central nervous system
├── brain/
│   ├── llm.py               # CognitiveEngine : LLM inference
│   ├── soul.py              # StateMachine : emotions & stream buffer
│   ├── autonomy_loop.py     # AutonomyEngine : proactive thought
│   ├── context.py           # Context management
│   └── work_mode.py         # Work mode prompt templates
├── vision/
│   ├── semantic.py          # SemanticSensor : UIA extraction
│   └── mss_capture.py       # VisionSensor : OCR fallback
├── sensors/
│   └── window_tracker.py    # WindowTracker : focus detection
├── audio/
│   ├── speaker.py           # VoiceEngine : TTS + DSP
│   └── ears.py              # EarsSensor : STT
├── physics/
│   └── kinematics.py        # KinematicsEngine : movement
├── ui/
│   ├── overlay.py           # WindowManager : transparent overlay
│   ├── face.py              # FaceRenderer : 60fps face
│   ├── chat.py              # ChatBubble : text overlay
│   ├── input_box.py         # InputBox : text input
│   ├── expressions.py       # Expression definitions
│   └── theme.py             # Visual theming
├── models/                  # Local GGUF models (gitignored)
├── voices/                  # Piper ONNX voice models
├── scripts/                 # Setup & download utilities
├── docs/
│   ├── ARCHITECTURE.md      # Detailed architecture docs
│   └── assets/              # Images & diagrams
├── .env                     # API keys (gitignored)
└── requirements.txt         # Python dependencies

Core Systems

Agentic Work Mode

Activated via Ctrl+Shift+Alt+W, Work Mode shifts B into a high-utility state:

• Semantic Monitoring : B monitors your progress on tasks in real-time.
• Contextual Curiosity : Proactively offers insights, documentation, or suggestions based on your current focus.
• Minimalist Presence : Dims facial expressions to minimize distraction while remaining vigilant.

flowchart TB
    A["User presses Ctrl+Shift+Alt+W"] --> B["B asks: what are you working on?"]
    B --> C["User defines goal"]
    C --> D["B monitors screen context"]
    D --> E{"Relevant content?"}
    E -- "Yes" --> F["B offers insight / help"]
    E -- "No" --> G["B stays silent"]
    F --> D
    G --> D

Hotkeys

Installation

1. Clone the repository:

   git clone https://github.com/Ahmad-Hassan-0/B---desktop-companion.git
   cd B---desktop-companion
   

2. Set up the environment:

   python -m venv venv
   venv\Scripts\activate      # Windows
   source venv/bin/activate   # Linux/macOS
   pip install -r requirements.txt
   

3. Configure API keys:

   cp .env.example .env
   # Edit .env with your Groq API key (get one at https://console.groq.com/)
   

4. Awaken B:

   python main.py
   

Safety

[!CAUTION]
Global Kill Switch: Ctrl+Shift+Alt+Q
This hotkey immediately terminates B and releases all system hooks. Use this if B becomes over-eager or if you need an instant exit.

Because B lives on a transparent, click-through overlay without a standard close button, the kill switch is the only way to exit. It is registered at the Win32 level and works even if the Qt event loop is unresponsive.

Technical Constraints