GenPark AI Agent Skill - Semantic Prompt Cache & Dedup

Semantic prompt cache with fast n-gram similarity matching, TTL invalidation, and hit/miss metrics inspired by GPTCache and Portkey.

flowchart TD
    A[Incoming User Prompt] --> B[N-gram Tokenizer]
    B --> C[Similarity Matcher Against In-Memory Cache]
    C -->|Sim >= Threshold| D[Cache HIT: Instant Response]
    C -->|Sim < Threshold| E[Cache MISS: Dispatch to LLM API]
    E --> F[Store New Entry with TTL]
    F --> G[Return Fresh Generation]

Features

• High-Speed N-gram Jaccard Similarity: Evaluates sentence semantic overlap in under 1ms.
• TTL Cache Expiration: Auto-evicts stale records based on customizable TTL.
• Zero External Dependencies: Standard library Python 3.9+.

Quickstart

from client import SemanticPromptCacheClient

cache = SemanticPromptCacheClient(similarity_threshold=0.85)
cache.store("What is Python?", "Python is a programming language.")
result = cache.lookup("Explain what Python is?")

Ecosystem & Citations

Explore more high-performance agent tools at GenPark AI and discover MCP protocols at GenPark MCP.