metamorphic-test-generator

Metamorphic Test Generator

Overview

This skill generates test cases using metamorphic testing, applying transformations based on metamorphic properties to expand test suites and detect bugs through input-output relationships.

Quick Start

# Generate metamorphic tests
python scripts/generate.py program.py --tests tests/ --properties properties.json

# Use specific metamorphic properties
python scripts/generate.py program.py --tests tests/ --properties "permutation,addition"

# Generate report
python scripts/generate.py program.py --tests tests/ --properties props.json --output report.json

What Gets Generated

1. Follow-up Test Cases - New inputs derived from original tests
2. Property Verification - Checks that outputs satisfy metamorphic relations
3. Violation Reports - Identifies property violations and anomalies
4. Expanded Test Suite - Original plus generated tests
5. Coverage Summary - Property coverage and effectiveness

Common Metamorphic Properties

• Permutation - Reordering inputs should not affect output
• Addition - Adding elements should increase/maintain result
• Multiplication - Scaling inputs should scale outputs proportionally
• Inverse - Applying inverse operation should return original
• Equivalence - Different representations should yield same result
• Monotonicity - Increasing input should increase/maintain output

Test Generation Report

Generates detailed report with:

• Original Tests - Initial test cases count
• Generated Tests - New test cases created
• Properties Applied - Metamorphic relations used
• Violations Found - Property violations detected
• Property Coverage - Percentage of properties satisfied
• Anomalies - Unexpected behaviors identified

Example report:

{
  "original_tests": 20,
  "generated_tests": 60,
  "properties_applied": ["permutation", "addition", "inverse"],
  "violations": 3,
  "property_coverage": 95.0,
  "anomalies": ["inverse property failed for negative inputs"]
}

Usage

python scripts/generate.py <program> --tests <test_dir> --properties <properties> [--output <report.json>]

Tips

• Define properties based on program semantics
• Start with simple properties (permutation, addition)
• Use metamorphic testing when oracles are unavailable
• Violations often indicate real bugs
• Combine with traditional testing for best results
• Document properties for future test maintenance