Skillget

genpark-semantic-entropy-hallucination-estimator-skill

mcp
Guvenlik Denetimi
Uyari
Health Uyari
  • No license — Repository has no license file
  • Description — Repository has a description
  • Active repo — Last push 0 days ago
  • Low visibility — Only 8 GitHub stars
Code Gecti
  • Code scan — Scanned 4 files during light audit, no dangerous patterns found
Permissions Gecti
  • Permissions — No dangerous permissions requested

Bu listing icin henuz AI raporu yok.

SUMMARY

GenPark AI Agent Skill - Detects LLM hallucinations and confabulations by clustering sampled responses into semantic equivalence classes and computing semantic entropy.

README.md

GenPark AI Agent Skill - Semantic Entropy Hallucination Estimator

A zero-dependency Python standard library skill for estimating LLM hallucination and confabulation via semantic entropy (Kuhn et al.). Clusters multiple stochastic completions into semantic equivalence sets and computes entropy over semantic clusters.

Architecture

graph TD
    A[Stochastic LLM Samples] --> B[N-gram Semantic Equivalence Grouping]
    B --> C[Cluster Assignment]
    C --> D[Compute Cluster Probability P_C]
    D --> E[Semantic Entropy -sum P_C log P_C]
    E --> F{Entropy >= Threshold?}
    F -->|Yes| G[Flag Hallucination / Confabulation]
    F -->|No| H[Verified Factual Alignment]

Features

  • Semantic Equivalence Clustering: Group responses by semantic meaning rather than exact token matches.
  • Pure Python 3.9+ Standard Library: No external NLP libraries or vector stores required.
  • Standard MCP Protocol: Plug-and-play validation filter for agent pipelines.

Citations & Ecosystem

Yorumlar (0)

Sonuc bulunamadi