🚀 AI Product Manager Toolkit

A comprehensive collection of prompts, templates, tools, and frameworks for Product Managers working with AI/ML products.

📚 What's Inside

🧭 Finding things

• By goal (recommended) → docs/tool-picker.md: what to open for planning, delivery, experiments, AI/ML work, and cost.
• Scripts → scripts/README.md: categorized index, one-line descriptions, and which sample CSV to use with each script.
• Prompts → prompts/README.md: index by category; files are prompts/*/*.prompt.md.
• Templates → templates/: PRD, OKR, RICE, technical spec, DX assessment, etc.
• MCPs → mcps/README.md: Jira, Confluence, GitHub, Slack, Notion, Braintrust, LangSmith, Product Analytics. Task-based routing → docs/tool-picker.md.
• Connectors → connectors/README.md: fetch.py --list for sources, fetch.py --describe issues for a column contract.
• Evals → evals/scripts/README.md: eval harness, regression runner, cost calculator.

🎯 Quick Start

Using Prompts

All prompt files end in .prompt.md. Copy the prompt content and use it with your preferred AI assistant (Claude, ChatGPT, Copilot, etc.).

# Example: Generate a PRD
cat prompts/core-pm/prd-generator.prompt.md

Using MCP Servers

MCP servers require Node.js 18+. See mcps/README.md for setup instructions.

cd mcps/servers/jira-pm-assistant
npm install
npm run build

Using Scripts

Python scripts require Python 3.10+. Sample data lives in scripts/samples/; see scripts/README.md for the full index and which sample file goes with each script.

pip install -r scripts/requirements.txt
python scripts/ab-test-calculator.py
python scripts/experiment-duration-calculator.py --baseline 0.05 --mde 0.10 --daily-visitors 5000
python scripts/ai-unit-economics-calculator.py --cost-per-request 0.002 --requests-per-month 1e6 --revenue-per-user 5 --mau 200000
python scripts/bedrock-cost-calculator.py --input-tokens 1000 --output-tokens 500 --model claude
python scripts/multi-model-cost-comparator.py --input-tokens 1000 --output-tokens 500
python scripts/model-selection-scorecard.py -s scripts/samples/sample-model-selection-scores.csv -w scripts/samples/sample-model-selection-weights.csv
python scripts/meeting-load-optimizer.py --csv scripts/samples/sample-meetings.csv
python scripts/feature-rollout-calculator.py --daily-volume 100000
python scripts/ai-initiative-roi-calculator.py --dev-cost 50000 --monthly-ai-cost 2000 --monthly-benefit 10000
python scripts/confidence-interval-calculator.py --n 500 --proportion 0.32
python scripts/nps-csat-summary.py nps --promoters 40 --passives 30 --detractors 30
python scripts/survey-sample-size.py --margin 0.05 --confidence 0.95
python scripts/latency-slo-calculator.py --availability 99.9 --requests-per-month 10e6
python scripts/churn-risk-calculator.py --cohort "New Users" --usage-drop 40 --adoption 25 --tickets 12
python scripts/prompt-cost-optimizer.py --file prompt.txt --model gpt-4o --requests-per-month 500000
python scripts/data-drift-detector.py --baseline baseline.csv --current current.csv
python scripts/adoption-funnel-analyzer.py --steps "Visit:10000" "Signup:4000" "Activate:2500" "Repeat:800"
python scripts/sla-uptime-calculator.py --sla 99.9 --incidents 45 120 15 --forecast-days 90
python scripts/velocity-trend-analyzer.py --sprints 38 42 35 45 40 48 --window 3 --target 45
python scripts/capacity-planning-calculator.py --team 6 --sprint-days 10 --pto 2 --meetings 0.2 --points-per-day 4
python scripts/cycle-lead-time-analyzer.py --csv tickets.csv --group-by type
python scripts/sprint-burndown-checker.py --csv burndown.csv --chart
python scripts/sprint-mix-report.py --csv sprint.csv --group-by type
python scripts/commitment-predictability-index.py --csv velocity.csv
python scripts/status-duration-analyzer.py --csv transitions.csv --chart
python scripts/experiment-result-interpreter.py --baseline 5.0 --variant 5.6 --n 8000
python scripts/backlog-aging-report.py --csv backlog.csv --oldest 15
python scripts/sprint-goal-checker.py --goals goals.csv --completed done.csv
python scripts/eval-label-economics.py --margin 0.05 --confidence 0.95 --proportion 0.5 --cost-per-label 2.50
python scripts/eval-score-trend.py --csv eval-runs.csv --chart
python scripts/incident-rate-trend.py --csv incidents.csv --chart
python scripts/risk-register-summary.py --csv risks.csv --top 5
python scripts/release-impact-summary.py --csv shipped.csv --version "v2.1.0" --bullets
python scripts/prompt-version-diff.py --old prompt_v1.txt --new prompt_v2.txt
python scripts/hallucination-safety-trend.py --csv evals.csv --metric-type hallucination --chart
python scripts/roadmap-timeline-summary.py --csv roadmap.csv --overlaps --by-quarter
python scripts/launch-readiness-score.py --csv checklist.csv --go-threshold 95
python scripts/feedback-theme-counter.py --csv feedback.csv --themes "pricing,reliability,ux,support"
python scripts/support-escalation-trend.py --csv tickets.csv --chart --group-by severity
python scripts/audit-checklist-summary.py --csv controls.csv --group-by domain --open
python scripts/budget-burn-summary.py --csv budget.csv --group-by category
python scripts/win-loss-summary.py --csv deals.csv --top 5 --group-by segment
python scripts/inference-latency-trend.py --csv latency-runs.csv --metric p99 --chart
python scripts/feature-adoption-trend.py --csv adoption.csv --chart --group-by segment
python scripts/stakeholder-signoff-tracker.py --csv signoffs.csv --pending --group-by deliverable
python scripts/dependency-blocked-summary.py --csv deps.csv --blocking
python scripts/beta-conversion-report.py --csv beta.csv --chart
python scripts/customer-health-score-trend.py --csv health.csv --at-risk-below 50 --chart --group-by segment
python scripts/release-cadence-report.py --csv releases.csv --period month --chart --group-by product
python scripts/agentic-cost-simulator.py --steps-per-task 6 --retry-rate 0.15 --tasks-per-month 20000
python scripts/agent-task-success-tracker.py --csv agent-runs.csv --target-success-rate 0.85
python scripts/guardrail-effectiveness-analyzer.py --csv guardrail-labels.csv --max-fn-rate 0.05
python scripts/red-team-coverage-tracker.py --csv redteam-results.csv
python scripts/model-deprecation-watch.py --csv model-portfolio.csv --warn-within-days 90
python scripts/model_pricing.py --check
python scripts/launch-checklist.py --name "Agent Copilot" --type backend --csv gate.csv

Trusting the numbers

• Pricing comes from one table (scripts/model_pricing.py),
  and every model carries a last_verified date. Anthropic rates are verified; Bedrock,
  OpenAI, and Google entries are inherited from earlier versions of this toolkit and are
  labeled UNVERIFIED until checked. Run python scripts/model_pricing.py --check to see
  what needs attention, and expect any comparison including an unverified model to say so.

• CSV headers don't have to match exactly. Duration (Minutes), duration_minutes,
  and DURATION MINUTES all resolve to the same column, so a raw Jira or analytics export
  works without hand-editing headers first.

• --output JSON carries the tool name, timestamp, and schema version alongside the
  result, so a number in a deck can be traced back to what produced it:
  python scripts/toolkit_io.py results.json.

• Every script runs, not just --helps. scripts/samples/ has real sample data for
  all 112 scripts, and scripts/smoke-test.py runs each one
  against it and checks it actually produces output: python scripts/smoke-test.py.
  Install requirements.txt first — 5 scripts need scipy
  or textblob and are reported as failures without them.

• Connectors are checked the same way, but end to end: python connectors/self-test.py
  fetches from a fixture and then runs every script the dataset contract claims to feed,
  failing if any of them reads zero rows.

• Where the data came from is recorded too. connectors/fetch.py writes a
  .meta.json sidecar beside every CSV it produces — source, column mapping, row count,
  timestamp — so a number traces back past the script to the query that fed it.

🗂️ Directory Structure

pm-toolkit/
├── README.md
├── .gitignore
├── prompts/
│   ├── core-pm/              # PRDs, user stories, stakeholder updates, pricing page, API changelog
│   ├── ai-ml/                # ML system design, model cards, MLOps, migration playbooks
│   └── developer-community/  # AI accelerator resources
├── templates/                # RICE, OKRs, technical specs, prompt library management
├── scripts/                  # See scripts/README.md for categorized index and sample CSVs
├── connectors/
│   ├── datasets.py           # Canonical dataset contracts (the column shapes scripts read)
│   ├── fetch.py              # CLI: fetch.py <source> <dataset> --out FILE
│   ├── base.py               # Shared config, retries, normalization, provenance sidecar
│   ├── sources/              # One module per system (csvfile, jira, gateway)
│   ├── profiles/             # Per-instance field mappings (e.g. Jira custom field IDs)
│   └── fixtures/             # Recorded responses for offline runs and self-test
├── frameworks/               # Prioritization, ML lifecycle, build vs. buy, deprecation playbook, SPACE
├── mcps/
│   ├── guides/               # Setup and use case documentation
│   ├── TOOLS.md              # Server → MCP tool name reference
│   └── servers/              # Jira, Confluence, GitHub, Slack, Notion, Calendar, etc.
├── agents/
│   ├── system-prompts/       # Ready-to-use agent personas
│   ├── rules/                # Cursor rules, Claude instructions
│   ├── evaluation/           # Agent evaluation frameworks
│   └── patterns/             # Agent design patterns
├── evals/
│   ├── frameworks/           # LLM evaluation methodology
│   ├── scripts/              # Eval harness, eval summary report generator
│   ├── templates/            # Eval planning docs
│   └── metrics/              # AI product metrics guide
├── learning/                 # AI/ML fundamentals, glossary, resources
├── strategy/                 # AI product strategy frameworks
├── governance/               # Responsible AI checklist
├── communication/            # Executive communication prompts
├── launch/                   # AI feature launch checklist
├── incidents/                # Incident response playbook
├── career/                   # AI PM skills roadmap
└── workflows/                # Daily AI PM workflow

☁️ Building on AWS Bedrock

If you use Amazon Bedrock, see:

• AWS Bedrock for PMs – concepts, model families, data/compliance
• scripts/bedrock-cost-calculator.py – estimate inference cost
• prompts/ai-ml/bedrock-model-selection.prompt.md – choose a model
• evals/frameworks/bedrock-eval-guide.md – run evals with Bedrock

🤝 Contributing

This toolkit is open for contributions. See CONTRIBUTING.md for where to add scripts, prompts, and templates so the repo stays easy to navigate. Then open a PR.

📄 License

MIT License - Use freely, and attribution appreciated.

Built for Product Managers navigating the AI era. 🧠