Receber por email
← Voltar para o catálogo

eval-outcomes

Holdout-safe Outcomes grading: project the locked eval substrate, then ingest one verdict.

393estrelas
Atualizado há 4 meses

Ver no GitHub ↗ Licença: NOASSERTION

Como adicionar

/plugin marketplace add boshu2/agentops

O comando exato pode variar conforme o repositório. Confira o README no GitHub.

Para o autor da skill

Cole no README do seu repo

Mostra que sua skill está catalogada na Skillteca, gera backlink e tráfego rastreável.

[![Skillteca](https://www.skillteca.com.br/api/badge/eval-outcomes/svg)](https://www.skillteca.com.br/skills/eval-outcomes)

Skills relacionadas

Ver todas de Desenvolvimento →

Alerta por categoria

Receba novas skills de Desenvolvimento toda segunda

1 email curto, só com as skills novas de Desenvolvimento. 4 minutos de leitura, sem spam, cancela com 1 clique.

Você confirma o email no primeiro envio. Sem spam. Cancela com 1 clique.

Ler descrição completa

$eval-outcomes — Outcomes as a Projection of the Locked Eval Substrate (Codex Native)

Quick Ref: Grade an agent's output via Outcomes (or any model) without forking the bar. ao eval outcomes compile projects a locked Task into a holdout-safe rubric (refuses to leak target/ground_truth — Managed Agents are not ZDR); grade it; ao eval outcomes ingest writes the score back as the one council verdict record. Outcomes is a projection, never an alternate authority.

Codex path

Codex has no Managed Agents loop. Call the same ao eval outcomes compile <input.json> to get a holdout-safe rubric, grade it locally (Inspect AI over the dev split, or the bushido llama.cpp Qwen grader over tailnet), then ao eval outcomes ingest <score.json> --json. Net: Codex never touches the cloud Outcomes API but produces a byte-identical verdict record.

Instructions

Load and follow the skill instructions from the sibling SKILL.md — OR read skills/eval-outcomes/SKILL.md in the host repo for the canonical specification. Then apply the three-phase Workflow (compile → grade → ingest), honoring the Critical Constraints (never send holdout target/ground_truth/PII; carry judge_content_hash; register a global Dolt burn for holdout grades).

Compartilhar X LinkedIn