Content Evaluation Framework

This skill provides a comprehensive, systematic rubric for evaluating educational book chapters and lessons with quantifiable quality standards.

Constitution Alignment: v4.0.1 emphasizing:

• Principle 1: Specification Primacy ("Specs Are the New Syntax")
• Section IIa: Panaversity 4-Layer Teaching Method
• Section IIb: AI Three Roles Framework (bidirectional co-learning)
• 8 Foundational Principles: Including Factual Accuracy, Coherent Structure, Progressive Complexity
• Nine Pillars (Section I): AI CLI, Markdown, MCP, AI-First IDEs, Cross-Platform, TDD, SDD, Composable Skills, Cloud-Native

Purpose

Evaluate educational content across 6 weighted categories to ensure:

• Technical correctness and code quality
• Effective pedagogical design and learning outcomes
• Clear, accessible writing for target audience
• Proper structure and organization
• AI-augmented learning principles (learning WITH AI, not generating FROM AI)
• Constitution compliance and standards adherence

When to Use This Skill

Invoke this evaluation framework at multiple checkpoints:

1. During Iterative Drafting - Mid-process quality checks to catch issues early
2. After Lesson/Chapter Completion - Comprehensive evaluation before moving to next content unit
3. On-Demand Review Requests - When user explicitly asks for quality assessment
4. Before Validation Phase - Part of the SDD Validate phase workflow for final sign-off

Evaluation Methodology

Scoring System

Multi-Tier Assessment:

• Excellent (90-100%) - Exceeds standards, exemplary quality
• Good (75-89%) - Meets all standards with minor improvements possible
• Needs Work (50-74%) - Meets some standards but requires significant revision
• Insufficient (<50%) - Does not meet minimum standards, requires major rework

Weighted Categories

The evaluation uses 6 categories with the following weights:

Total Weighted Score Calculation:

Final Score = (Technical × 0.30) + (Pedagogical × 0.25) + (Writing × 0.20) +
              (Structure × 0.15) + (AI-First × 0.10)

Constitution Compliance: Must achieve "Pass" status. If "Fail," content cannot proceed regardless of weighted score.

How to Conduct an Evaluation

Step 1: Prepare Context

Before evaluation, gather:

• Content being evaluated (lesson.md, chapter.md, or section file)
• Relevant spec, plan, and tasks files from specs/<feature>/
• Constitution file (.specify/memory/constitution.md)
• Learning objectives and success criteria for the content unit
• Output style template used (.claude/output-styles/lesson.md or similar)

Step 2: Load Detailed Rubric

Read the detailed tier criteria for each category:

Read: references/rubric-details.md

This file contains specific criteria defining Excellent/Good/Needs Work/Insufficient for each of the 6 categories.

Step 3: Evaluate Constitution Compliance First

Constitution compliance is a gate - if content fails constitutional requirements, it cannot proceed.

Use the constitution checklist:

Read: references/constitution-checklist.md

Assess all non-negotiable principles and requirements. Mark as Pass or Fail with specific violations noted.

If Constitution Compliance = Fail: Stop evaluation and report violations immediately. Content must be revised before proceeding.

If Constitution Compliance = Pass: Continue to weighted category evaluation.

Step 4: Score Each Weighted Category

For each of the 5 weighted categories (Technical Accuracy, Pedagogical Effectiveness, Writing Quality, Structure & Organization, AI-First Teaching):

1. Review specific criteria from rubric-details.md for that category
2. Assess content against criteria for each tier
3. Assign tier (Excellent/Good/Needs Work/Insufficient) with score range
4. Record specific evidence - Quote examples, note line numbers, cite specific passages
5. Provide improvement recommendations - Concrete, actionable feedback

Step 5: Calculate Weighted Score

Apply the weighted formula:

Final Score = (Technical × 0.30) + (Pedagogical × 0.25) + (Writing × 0.20) +
              (Structure × 0.15) + (AI-First × 0.10)

Convert tier scores to numeric values:

• Excellent: 95%
• Good: 82%
• Needs Work: 62%
• Insufficient: 40%

(Or use specific numeric score within tier range if warranted)

Step 6: Generate Evaluation Report

Use the structured evaluation template:

Read: references/evaluation-template.md

Complete all sections:

1. Executive Summary - Overall score, tier, pass/fail status
2. Category Scores - Table showing each category score, tier, and weight contribution
3. Detailed Findings - Evidence-based assessment for each category
4. Strengths - What the content does well (specific examples)
5. Areas for Improvement - Prioritized list of issues with recommendations
6. Constitution Compliance Status - Pass/Fail with specific principle checks
7. Actionable Next Steps - Concrete tasks to improve content

Step 7: Communicate Results

Present evaluation report with:

• Clear verdict - Pass/Fail and overall quality tier
• Evidence-based feedback - Specific quotes and line numbers
• Prioritized improvements - Most critical issues first
• Encouragement - Acknowledge strengths and effort

Evaluation Best Practices

Be Objective and Evidence-Based

• Quote specific passages from content being evaluated
• Reference line numbers or section headers
• Compare against objective rubric criteria, not subjective preference
• Use concrete metrics where possible (word count, readability scores, etc.)

Focus on Standards, Not Perfection

• Content rated "Good" (75-89%) is publication-ready with minor polish
• Content rated "Excellent" (90-100%) exceeds standards but is not required
• Focus improvements on moving "Needs Work" → "Good" before "Good" → "Excellent"

Provide Actionable Feedback

• Don't just say "improve clarity" - specify which sentences are unclear and suggest rewrites
• Don't just say "add examples" - suggest specific example types that would help
• Prioritize recommendations: critical (blocking issues) → important → nice-to-have

Respect the Learning Journey

• Recognize iterative improvement - drafts evolve through multiple passes
• Celebrate progress and strengths
• Frame criticism constructively as opportunities for growth
• Remember: the goal is helping create excellent educational content, not gatekeeping

Quality Gates and Thresholds

Minimum Acceptance Threshold

• Constitution Compliance: MUST be Pass (gate)
• Overall Weighted Score: MUST be ≥ 75% (Good or better)
• No category below 50%: Each individual category must achieve at least "Needs Work" tier

Recommended for Publication

• Constitution Compliance: Pass
• Overall Weighted Score: ≥ 82% (Good tier)
• Technical Accuracy: ≥ 75% (Good tier) - Critical for credibility
• Pedagogical Effectiveness: ≥ 75% (Good tier) - Critical for learning outcomes

Exemplary Content (Optional)

• **Overall Weighted Score