python-regression-test-generator

Python Regression Test Generator

Overview

This skill automatically generates regression tests for Python code by analyzing differences between old and new versions, migrating existing tests, and creating new tests for modified or added functionality. It ensures previously tested behavior still works while covering new code paths.

Test Generation Workflow

Follow these steps to generate regression tests:

1. Analyze Code Changes

Compare old and new versions to identify:

Function Signature Changes:

• Added parameters (with or without defaults)
• Removed parameters
• Changed parameter types or names
• Changed return types

Logic Changes:

• Modified implementation
• New validation rules
• Changed error handling
• Algorithm improvements

Structural Changes:

• Added functions/classes/methods
• Removed functions/classes/methods
• Renamed functions/classes/methods
• Moved code between modules

Example Analysis:

# Old version
def calculate_total(items):
    return sum(item.price for item in items)

# New version
def calculate_total(items, tax_rate=0.0):
    subtotal = sum(item.price for item in items)
    return subtotal * (1 + tax_rate)

Changes identified:

• Added parameter: tax_rate with default value 0.0
• Modified logic: now applies tax calculation
• Backward compatible: old calls still work

2. Analyze Existing Tests

Review old tests to understand:

• Test structure (unittest vs pytest)
• Test coverage (which functions/paths are tested)
• Test patterns (fixtures, mocks, parametrization)
• Dependencies and setup requirements

Example Existing Test:

def test_calculate_total():
    items = [Item(price=10), Item(price=20)]
    assert calculate_total(items) == 30

Analysis:

• Uses pytest style
• Tests basic functionality
• No mocking needed
• Simple assertion

3. Migrate Existing Tests

Update tests to work with new code:

For backward-compatible changes:

• Keep original test (ensures backward compatibility)
• Add new tests for new functionality

For breaking changes:

• Update test to match new signature/behavior
• Document what changed in test docstring

Migrated Tests:

def test_calculate_total_no_tax():
    """Regression: ensure backward compatibility with no tax"""
    items = [Item(price=10), Item(price=20)]
    assert calculate_total(items) == 30  # Uses default tax_rate=0.0

def test_calculate_total_with_tax():
    """New: test tax calculation functionality"""
    items = [Item(price=10), Item(price=20)]
    assert calculate_total(items, tax_rate=0.1) == 33.0

4. Generate Tests for New Functionality

Create tests for:

• New parameters and their effects
• New functions/methods
• New code paths
• Edge cases and boundaries
• Error conditions

Generated Tests:

def test_calculate_total_zero_tax():
    """Test explicit zero tax rate"""
    items = [Item(price=100)]
    assert calculate_total(items, tax_rate=0.0) == 100

def test_calculate_total_high_tax():
    """Test high tax rate"""
    items = [Item(price=100)]
    assert calculate_total(items, tax_rate=0.5) == 150

def test_calculate_total_empty_items():
    """Edge case: empty items list"""
    assert calculate_total([], tax_rate=0.1) == 0

def test_calculate_total_negative_tax():
    """Edge case: negative tax (discount)"""
    items = [Item(price=100)]
    assert calculate_total(items, tax_rate=-0.1) == 90

5. Create Mocks for Dependencies

Identify dependencies that need mocking:

• External APIs and HTTP requests
• Database operations
• File I/O operations
• Email/notification services
• Time-dependent operations
• Random number generation

Example with External Dependency:

New Code:

def send_notification(user_id, message):
    user = database.get_user(user_id)
    email_service.send_email(
        to=user.email,
        subject='Notification',
        body=message
    )
    return True

Generated Test with Mocks:

from unittest.mock import Mock, patch

def test_send_notification():
    """Test notification sending with mocked dependencies"""
    # Mock database
    mock_db = Mock()
    mock_user = Mock()
    mock_user.email = 'user@example.com'
    mock_db.get_user.return_value = mock_user

    # Mock email service
    with patch('myapp.email_service.send_email') as mock_send:
        mock_send.return_value = True

        with patch('myapp.database', mock_db):
            result = send_notification(user_id=123, message="Hello")

            assert result is True
            mock_db.get_user.assert_called_once_with(123)
            mock_send.assert_called_once_with(
                to='user@example.com',
                subject='Notification',
                body='Hello'
            )

6. Add Setup and Teardown

Determine if tests need:

• Shared fixtures
• Database setup/cleanup
• File creation/deletion
• State initialization

unittest Setup:

class TestCalculateTotal(unittest.TestCase):
    def setUp(self):
        """Run before each test"""
        self.items = [
            Item(price=10),
            Item(price=20),
            Item(price=30)
        ]

    def test_no_tax(self):
        result = calculate_total(self.items)
        self.assertEqual(result, 60)

    def test_with_tax(self):
        result = calculate_total(self.items, tax_rate=0.1)
        self.assertAlmostEqual(result, 66.0)

pytest Fixtures:

@pytest.fixture
def items():
    """Fixture providing test items"""
    return [
        Item(price=10),
        Item(price=20),
        Item(price=30)
    ]

def test_no_tax(items):
    result = calculate_total(items)
    assert result == 60

def test_with_tax(items):
    result = calculate_total(items, tax_rate=0.1)
    assert result == pytest.approx(66.0)

7. Ensure Test Quality

Verify generated tests are:

• Deterministic: Same input always produces same output
• Isolated: Tests don't depend on each other
• Readable: Clear test names and docstrings
• Maintainable: Follow project conventions
• Executable: Can run without errors

Complete Example

Scenario: Adding Tax Calculation

Old Code (calculator.py):

class Calculator:
    def calculate_total(self, items):
        """Calculate total price of items"""
        return sum(item.price for item in items)

Old Test (test_calculator.py):

import unittest
from calculator import Calculator
from models import Item

class TestCalculator(unittest.TestCase):
    def test_calculate_total(self):
        calc = Calculator()
        items = [Item(price=10), Item(price=20)]
        result = calc.calculate_total(items)
        self.assertEqual(result, 30)

New Code (calculator.py):

class Calculator:
    def calculate_total(self, items, tax_rate=0.0, discount=0.0):
        """Calculate total price with tax and discount"""
        if tax_rate < 0 or tax_rate > 1:
            raise ValueError("Tax rate must be between 0 and 1")
        if discount < 0 or discount > 1:
            raise ValueError("Discount must be between 0 and 1")

        subtotal = sum(item.price for item in items)
        after_discount = subtotal * (1 - discount)
        total = after_discount * (1 + tax_rate)
        return round(total, 2)

Generated Regression Tests (test_calculator.py):

import unittest
from unittest.mock import Mock
from calculator import Calculator
from models import Item

class TestCalculator(unittest.TestCase):
    def setUp(self):
        """Set up test fixtures"""
        self.calc = Calculator()
        self.items = [Item(price=10), Item(price=20)]

    def test_calculate_total_backward_compatibility(self):
        """Regression: ensure old behavior still works"""
        result = self.calc.calculate_total(self.items)
        self.assertEqual(result, 30)

    def test_calculate_total_with_tax(self):
        """New: test ta