dspy-evaluate

Name: dspy-evaluate
Rating: 5 (6 reviews)
Author: lebsral

Use when you need to measure how well your DSPy program performs — writing metrics, scoring against a dev set, or comparing before/after optimization. Common scenarios - measuring accuracy before and after optimization, writing custom metrics for your task, scoring a program against a held-out dev set, comparing two prompt strategies, building a test suite for AI quality, or running regression tes

6stars

Updated 13 days ago

View on GitHub ↗

How to add

/plugin marketplace add lebsral/DSPy-Programming-not-prompting-LMs-skills

The exact command may vary by repository. Check the README on GitHub.

For the skill author

Drop this on your repo README

Shows your skill is listed on Skillteca, generates a backlink and trackable traffic.

[![Listada na Skillteca](https://www.skillteca.com.br/api/badge/dspy-evaluate/svg)](https://www.skillteca.com.br/skills/dspy-evaluate?utm_source=badge&utm_medium=readme&utm_campaign=badge)

#ai #test

Related skills

See all in Desenvolvimento →

claude-api

143.8k

Build, debug, and optimize Claude API / Anthropic SDK apps. Apps built with this skill should include prompt caching. Also handles migrating existing Claude API code between Claude model versions (4.5 → 4.6, 4.6 → 4.7, retired-model replacements). TRIGGER when: code imports `anthropic`/`@anthropic-ai/sdk`; user asks for the Claude API, Anthropic SDK, or Managed Agents; user adds/modifies/tunes a C

Desenvolvimento#ai#apiby anthropics

skill-creator

143.8k

Create new skills, modify and improve existing ones, and measure their performance. Use this skill for developing, editing, optimizing, testing, and benchmarking skills, as well as refining their descriptions for better triggering.

Desenvolvimento#testby anthropics

oh-my-issues

79.7k

This skill clusters GitHub issue backlogs by root cause into plan-master issues, redirects related children, and bundles architectural-fix PRs to close clusters atomically. It's ideal for triaging and consolidating numerous issues sharing underlying defects, or for building a plan series or roadmap.

Desenvolvimento#github#gitby thedotmack

claude-mem

79.7k

Captures agent actions across sessions, compresses them with AI, and injects relevant context into future interactions. Compatible with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode, and more.

Desenvolvimento#aiby thedotmack

Category alert

Get new Desenvolvimento skills every Monday

One short email with only the new Desenvolvimento skills. 4 minutes of reading, no spam, unsubscribe with one click.

You confirm your email on the first send. No spam. Unsubscribe with one click.

Evaluate Your DSPy Program

Guide the user through measuring AI quality with DSPy's Evaluate class. The pattern: pick a metric, prepare a devset, run the evaluator, interpret results, then feed the same metric into an optimizer.

What is dspy.Evaluate

dspy.Evaluate runs your program on every devset example, scores each with a metric, and reports the aggregate score. It handles threading and progress display. Returns a percentage (0-100).

Built-in metrics

DSPy provides `answer_exac

[Description truncada. Veja o README completo no GitHub.]

ShareX LinkedIn

Comments · No comments

No comments yet. Be the first.