Skillget

multimodal-agent-workshop

agent
Security Audit
Pass
Health Pass
  • License — License: MIT
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Community trust — 16 GitHub stars
Code Pass
  • Code scan — Scanned 1 files during light audit, no dangerous patterns found
Permissions Pass
  • Permissions — No dangerous permissions requested

No AI report is available for this listing yet.

SUMMARY

🖼️ Workshop: Build a multimodal AI agent with Haystack & GPT-4o — featuring image understanding, document retrieval, conversational memory, and human-in-the-loop safety controls

README.md

🖼️ Giving Eyes to Your AI: Engineering a Multimodal Agent

A hands-on workshop exploring multimodal AI agents with Haystack.

What You'll Build

  • 📄 Multimodal indexing pipeline (PDFs + images) using CLIP embeddings
  • 🤖 Vision-enabled agent powered by GPT-4o
  • 🔍 RAG tool for searching company policies
  • 💬 Conversational memory for context-aware interactions
  • 🔁 Human-in-the-loop controls for sensitive actions

Get Started

👉 See multimodal_agent_notebook.ipynb for the full interactive experience.

Deploy with Hayhooks

Want to deploy the agent as an API? Check out multimodal-agent/pipeline_wrapper.py — a Python script version of the notebook with Hayhooks integration pre-configured for serving the conversational agent.

Files

The files/ directory contains the sample data used in the workshop:

  • receipt.jpeg — A sample receipt image for the expense reimbursement demo
  • social_budget_policy.md — Company policy document for retrieval

Requirements

  • Python 3.10+
  • OpenAI API key (or your preferred LLM provider)
  • See the notebook for full package installation instructions

Reviews (0)

No results found