This skill should be used when the user asks to "evaluate agent performance", "build test framework", "measure agent quality", "create evaluation rubrics", "implement LLM-as-judge", "compare model outputs", "mitigate evaluation bias", or mentions multi-dimensional evaluation, agent testing, quality gates, direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated qualit
Receba novas skills de Design e Frontend toda segunda
1 email curto, só com as skills novas de Design e Frontend. 4 minutos de leitura, sem spam, cancela com 1 clique.
Você confirma o email no primeiro envio. Sem spam. Cancela com 1 clique.
Ler descrição completaRecolher descrição
Evaluating LLM Agent Systems
Agent evaluation requires fundamentally different approaches than traditional software testing. Agents make dynamic decisions, are non-deterministic, and often lack single correct answers. Effective evaluation must account for these characteristics while providing actionable feedback.
Key insight: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known
[Description truncada. Veja o README completo no GitHub.]
