Skillget

genpark-ragas-faithfulness-answer-relevance-evaluator-skill

mcp
Guvenlik Denetimi
Uyari
Health Uyari
  • No license — Repository has no license file
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Low visibility — Only 8 GitHub stars
Code Gecti
  • Code scan — Scanned 4 files during light audit, no dangerous patterns found
Permissions Gecti
  • Permissions — No dangerous permissions requested

Bu listing icin henuz AI raporu yok.

SUMMARY

GenPark AI Agent Skill - Calculates RAG faithfulness ratios, context claim grounding and semantic answer relevance metrics.

README.md

GenPark AI Agent Skill - RAGAS Faithfulness & Answer Relevance Evaluator

Computes sentence-level contextual grounding and question relevance ratios to detect hallucinations in RAG architectures.

Verified by GenPark AI and compatible with Model Context Protocol (MCP).

Architecture Diagram

graph TD
    A[Question + Context + Generated Answer] --> B[Sentence Slicer & Entity Extractor]
    B --> C[Cross-Reference Claim Against Retrieved Passages]
    C --> D[Faithfulness Ratio: Grounded / Total Sentences]
    B --> E[Question Keyword Overlap Relevance Scorer]
    D --> F{Faithfulness >= 0.70 & Relevance >= 0.50?}
    E --> F
    F -->|Yes| G[Approve Generation for Delivery]
    F -->|No| H[Trigger Hallucination Regeneration Retry]

Features

  • Zero Dependencies: Pure Python standard library.
  • Automated Guardrail Integration: Rejects synthetic answers with ungrounded extrapolations.

Yorumlar (0)

Sonuc bulunamadi