evaluating-code-models

Name: evaluating-code-models
Rating: 5 (4 reviews)
Author: immacualate

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks using pass@k metrics. This industry standard from the BigCode Project, used by HuggingFace leaderboards, is ideal for benchmarking code models and measuring code generation quality.

4stars

Updated 13 days ago

View on GitHub ↗License: MIT

How to add

/plugin marketplace add immacualate/claude-forge

The exact command may vary by repository. Check the README on GitHub.

For the skill author

Drop this on your repo README

Shows your skill is listed on Skillteca, generates a backlink and trackable traffic.

[![Listada na Skillteca](https://www.skillteca.com.br/api/badge/evaluating-code-models/svg)](https://www.skillteca.com.br/skills/evaluating-code-models?utm_source=badge&utm_medium=readme&utm_campaign=badge)

#test

Related skills

See all in Desenvolvimento →

claude-api

143.8k

Build, debug, and optimize Claude API / Anthropic SDK apps. Apps built with this skill should include prompt caching. Also handles migrating existing Claude API code between Claude model versions (4.5 → 4.6, 4.6 → 4.7, retired-model replacements). TRIGGER when: code imports `anthropic`/`@anthropic-ai/sdk`; user asks for the Claude API, Anthropic SDK, or Managed Agents; user adds/modifies/tunes a C

Desenvolvimento#ai#apiby anthropics

skill-creator

143.8k

Create new skills, modify and improve existing ones, and measure their performance. Use this skill for developing, editing, optimizing, testing, and benchmarking skills, as well as refining their descriptions for better triggering.

Desenvolvimento#testby anthropics

oh-my-issues

79.7k

This skill clusters GitHub issue backlogs by root cause into plan-master issues, redirects related children, and bundles architectural-fix PRs to close clusters atomically. It's ideal for triaging and consolidating numerous issues sharing underlying defects, or for building a plan series or roadmap.

Desenvolvimento#github#gitby thedotmack

claude-mem

79.7k

Captures agent actions across sessions, compresses them with AI, and injects relevant context into future interactions. Compatible with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode, and more.

Desenvolvimento#aiby thedotmack

Category alert

Get new Desenvolvimento skills every Monday

One short email with only the new Desenvolvimento skills. 4 minutes of reading, no spam, unsubscribe with one click.

You confirm your email on the first send. No spam. Unsubscribe with one click.

BigCode Evaluation Harness - Code Model Benchmarking

Quick Start

BigCode Evaluation Harness evaluates code generation models across 15+ benchmarks including HumanEval, MBPP, and MultiPL-E (18 languages).

Installation:

git clone https://github.com/bigcode-project/bigcode-evaluation-harness.git
cd bigcode-evaluation-harness
pip install -e .
accelerate config

Evaluate on HumanEval:

accelerate launch main.py \
  --model bigcode/starcoder2-7b \
  --tasks humaneval

[Description truncada. Veja o README completo no GitHub.]

ShareX LinkedIn

Comments · No comments

No comments yet. Be the first.