characterization-test-generator

Characterization Test Generator Skill

Generates characterization tests (also known as golden master tests or approval tests) to capture existing system behavior before migration, ensuring functional equivalence after changes.

Purpose

Enable behavior preservation during migration through:

• Golden master test creation
• Approval test generation
• Edge case discovery
• Input/output recording
• Behavior snapshot capture
• Regression baseline establishment

Capabilities

1. Golden Master Test Creation

• Capture current system outputs for given inputs
• Store outputs as reference "golden" files
• Generate comparison test infrastructure
• Support multiple output formats
• Enable baseline versioning

2. Approval Test Generation

• Create human-reviewable test outputs
• Generate diff-based comparison tests
• Support approval workflows
• Enable incremental approval
• Track approval history

3. Edge Case Discovery

• Analyze code paths for boundary conditions
• Generate boundary value inputs
• Identify error handling scenarios
• Map exception paths
• Create comprehensive test coverage

4. Input/Output Recording

• Instrument code for I/O capture
• Record API request/response pairs
• Capture database interactions
• Log external service calls
• Store timing and sequence data

5. Behavior Snapshot

• Capture state at key points
• Record side effects
• Document non-deterministic behavior
• Identify environmental dependencies
• Map configuration impacts

6. Regression Baseline Establishment

• Create baseline test suites
• Define acceptance thresholds
• Configure tolerance levels
• Set up CI/CD integration
• Generate coverage reports

Tool Integrations

This skill can leverage the following external tools when available:

| Tool | Purpose | Integration Method | |------|---------|-------------------| | ApprovalTests | Approval testing framework | Library | | Jest snapshots | JavaScript snapshot testing | Framework | | pytest-snapshot | Python snapshot testing | Plugin | | TextTest | Golden master testing | CLI | | Verify | .NET approval testing | Library | | Scientist | Safe refactoring library | Library | | AI Testing MCP | AI-powered test generation | MCP Server |

Usage

Basic Test Generation

# Invoke skill for characterization test generation
# The skill will analyze code and generate tests

# Expected inputs:
# - targetPath: Path to code to characterize
# - testFramework: 'jest' | 'pytest' | 'junit' | 'nunit' | 'auto'
# - outputDir: Directory for generated tests
# - captureMode: 'snapshot' | 'approval' | 'recording'

Generation Workflow

1. Analysis Phase
2. Identify testable units (functions, methods, endpoints)
3. Map input parameters and types
4. Detect output formats
5. Find external dependencies

1. Input Discovery Phase
2. Extract existing test inputs
3. Generate boundary values
4. Identify representative cases
5. Create combinatorial inputs

1. Capture Phase
2. Execute code with inputs
3. Record outputs and side effects
4. Capture state changes
5. Log external interactions

1. Test Generation Phase
2. Generate test code structure
3. Create golden master files
4. Set up comparison logic
5. Configure CI integration

Output Schema

{
  "generationId": "string",
  "timestamp": "ISO8601",
  "target": {
    "path": "string",
    "language": "string",
    "framework": "string",
    "unitsAnalyzed": "number"
  },
  "testsGenerated": {
    "total": "number",
    "byType": {
      "snapshot": "number",
      "approval": "number",
      "recording": "number"
    },
    "coverage": {
      "functions": "number",
      "branches": "number",
      "lines": "number"
    }
  },
  "characterizations": [
    {
      "unit": "string",
      "type": "function|method|endpoint|class",
      "inputs": [
        {
          "name": "string",
          "type": "string",
          "values": ["any"],
          "source": "existing|generated|boundary"
        }
      ],
      "outputs": {
        "type": "string",
        "goldenMasterPath": "string",
        "checksum": "string"
      },
      "testFile": "string",
      "baselineApproved": "boolean"
    }
  ],
  "edgeCases": [
    {
      "unit": "string",
      "case": "string",
      "input": "any",
      "expectedBehavior": "string",
      "covered": "boolean"
    }
  ],
  "dependencies": {
    "external": ["string"],
    "mocked": ["string"],
    "recorded": ["string"]
  },
  "artifacts": {
    "testSuite": "string",
    "goldenMasters": "string",
    "recordings": "string",
    "coverageReport": "string"
  }
}

Integration with Migration Processes

This skill integrates with the following Code Migration/Modernization processes:

• migration-testing-strategy: Primary tool for test baseline creation
• code-refactoring: Ensure behavior preservation during refactoring
• framework-upgrade: Verify functionality after upgrade
• monolith-to-microservices: Validate service extraction

Configuration

Skill Configuration File

Create .characterization-tests.json in the project root:

{
  "testFramework": "auto",
  "outputDir": "./tests/characterization",
  "captureMode": "snapshot",
  "goldenMasterDir": "./tests/golden-masters",
  "inputGeneration": {
    "boundary": true,
    "combinatorial": true,
    "maxCombinations": 100,
    "includeNulls": true,
    "includeEmpty": true
  },
  "outputCapture": {
    "format": "json",
    "normalizeWhitespace": true,
    "ignorePaths": ["$.timestamp", "$.requestId"],
    "tolerance": {
      "numeric": 0.001,
      "dateTime": "1s"
    }
  },
  "dependencies": {
    "mockExternal": true,
    "recordMode": false,
    "replayMode": true
  },
  "approval": {
    "requireApproval": true,
    "approvalDir": "./tests/approvals",
    "reportFormat": "markdown"
  },
  "ci": {
    "failOnNewTests": false,
    "updateGoldenOnPass": false,
    "generateCoverageReport": true
  }
}

MCP Server Integration

When AI Testing MCP Server is available:

// Example AI-powered test generation
{
  "tool": "ai_testing_generate",
  "arguments": {
    "target": "./src/services/user.ts",
    "framework": "jest",
    "style": "characterization"
  }
}

Test Patterns

Snapshot Testing (Jest)

// Generated characterization test
describe('UserService', () => {
  describe('calculateDiscount', () => {
    it('should match snapshot for standard customer', () => {
      const result = userService.calculateDiscount({
        customerId: 'C001',
        purchaseAmount: 100,
        loyaltyPoints: 500
      });
      expect(result).toMatchSnapshot();
    });

    it('should match snapshot for premium customer', () => {
      const result = userService.calculateDiscount({
        customerId: 'C002',
        purchaseAmount: 100,
        loyaltyPoints: 5000,
        isPremium: true
      });
      expect(result).toMatchSnapshot();
    });

    // Edge cases
    it('should match snapshot for zero amount', () => {
      const result = userService.calculateDiscount({
        customerId: 'C001',
        purchaseAmount: 0,
        loyaltyPoints: 0
      });
      expect(result).toMatchSnapshot();
    });
  });
});

Approval Testing (ApprovalTests)

// Generated approval test
public class UserServiceCharacterizationTest {

    @Test
    public void calculateDiscount_standardCustomer() {
        UserService service = new UserService();
        DiscountResult result = service.calculateDiscount(
            new DiscountRequest("C001", 100.0, 500)
        );
        Approvals.verify(result);
    }

    @Test
    public void calculateDiscount_boundaryValues() {
        UserService service = new UserService();

        // Boundary: minimum values
        Approvals.verify(service.calculateDiscount(
            new DiscountRequest("C001", 0.01, 0)
        ), "minimum");

        // Boundary: maximum values
        Approvals.verify(serv