llm-evaluation-engineering

LLM-evaluation engineering team — 2 agents (eval-strategy-lead, eval-harness-engineer) for the question every team shipping an AI feature must answer: is it getting better or worse? Eval design (task-grounded metrics not leaderboard scores, offline-vs-online split, sample size, a ship-gate decided as a number before the run) and the harness that proves it (frozen golden sets with provenance, LLM-as-judge rubrics with a bias audit + human calibration, CI regression gates, guardrail/red-team suites). 3 skills, a decision-tree knowledge bank (an eval-method Mermaid tree + a dated 2026 tooling/method map), 5 best-practices. Engineering decision-support, not a safety certification; model/judge/tooling specifics are volatile, retrieval-dated + verify-at-use; no eval-data PII. Seams: ai-rag-engineering (retrieval quality), ai-coding-model-guidance (model pick), ml-engineering (classical training), experimentation-growth-engineering (product A/B). Requires ravenclaude-core@>=0.7.0.