test-case-documentation

Test Case Documentation

Overview

Generate comprehensive, structured documentation for test suites by analyzing test code, framework output, and source context to produce both overview summaries and detailed test case specifications.

Workflow

1. Understand Documentation Needs

Determine what documentation is needed:

Questions to ask:

• What test suites need documentation?
• Is this for developers, QA, or stakeholders?
• Focus on overview or detailed specifications?
• Include coverage analysis?

Identify test location:

# Find test files
find . -name "test_*.py" -o -name "*_test.py"

# Count test files
find . -name "test_*.py" | wc -l

# Check test framework
grep -r "import pytest\|import unittest" tests/

2. Extract Test Information

Gather test data from multiple sources.

Source 1: Test Code Analysis

Use the bundled script to extract test metadata:

# Extract tests from directory
python scripts/extract_tests.py /path/to/tests

# Exclude specific directories
python scripts/extract_tests.py /path/to/tests venv,.venv,__pycache__

What it extracts:

• Test function names
• Test classes and organization
• Docstrings
• Test types (unit, integration, e2e)
• Pytest markers/tags
• File locations and line numbers

Manual extraction:

Read test files to understand:

def test_user_registration_with_valid_data():
    """
    Test successful user registration with valid input.

    Given: Valid email, username, and password
    When: User submits registration form
    Then: User account is created and welcome email sent
    """
    # Extract: purpose, preconditions, expected behavior
    user_data = {"email": "test@example.com", ...}  # Extract: test data
    result = service.register(user_data)  # Extract: operation
    assert result.success  # Extract: assertions/expected results
    assert result.user_id > 0

Key information to extract:

• Test purpose (from name and docstring)
• Test data (sample inputs)
• Operations performed
• Assertions (expected outcomes)
• Setup and teardown (fixtures)

Source 2: Test Framework Output

Capture test execution results:

# Run tests with verbose output
pytest tests/ -v > test_output.txt

# Run with detailed output
pytest tests/ -v --tb=short > test_results.txt

# Generate coverage report
pytest tests/ --cov=src --cov-report=term-missing > coverage.txt

Extract from output:

• Which tests passed/failed
• Execution time
• Coverage percentages
• Missing coverage areas

Source 3: Existing Test Documentation

Check for existing docs:

# Look for test documentation
find . -name "*test*.md" -o -name "TEST*.md"

# Check for test plans
find . -name "*test*plan*.md"

# Look for docstrings in conftest.py
cat tests/conftest.py

Source 4: Source Code Context

Understand what's being tested:

# Find source files related to tests
# test_user_service.py -> user_service.py
ls src/user_service.py

# Read source to understand functionality
cat src/user_service.py

3. Organize Test Information

Structure tests hierarchically.

See [documentation-templates.md](references/documentation-templates.md) for detailed templates.

Organization structure:

Test Suite Overview
├── Summary Statistics
├── Test Organization (directory structure)
├── Coverage Summary
└── Test Files
    ├── File 1: test_user_service.py
    │   ├── TestUserRegistration (class)
    │   │   ├── test_valid_registration (unit)
    │   │   ├── test_duplicate_email (unit)
    │   │   └── test_weak_password (unit)
    │   └── TestUserAuthentication (class)
    │       ├── test_login_success (unit)
    │       └── test_login_failure (unit)
    └── File 2: test_api.py
        └── test_full_workflow (integration)

Categorize by:

• Test type (unit, integration, e2e)
• Module/feature tested
• Priority/criticality
• Test status (passing, failing, skipped)

4. Generate Documentation

Create structured markdown documentation.

Level 1: Overview Documentation

High-level summary for project understanding:

# Test Suite: User Management

**Last Updated:** 2026-02-15
**Total Tests:** 42
**Coverage:** 87%
**Status:** ✅ 40 passing, ❌ 2 failing

## Summary

Comprehensive test suite for user management functionality including registration,
authentication, profile management, and user data export.

## Test Statistics

- **Unit Tests:** 28 (67%)
- **Integration Tests:** 12 (28%)
- **End-to-End Tests:** 2 (5%)

## Test Organization

tests/ ├── unit/ │ ├── test_user_service.py (15 tests) │ ├── test_auth_service.py (8 tests) │ └── test_validators.py (5 tests) ├── integration/ │ ├── test_api.py (10 tests) │ └── test_workflows.py (2 tests) └── e2e/ └── test_complete_flows.py (2 tests)

## Coverage by Module

| Module | Coverage | Tests | Priority |
|--------|----------|-------|----------|
| user_service.py | 95% | 15 | High |
| auth_service.py | 88% | 8 | High |
| validators.py | 94% | 5 | Medium |

## Quick Start

Run all tests

pytest tests/

Run unit tests only

pytest tests/unit/

Run with coverage

pytest --cov=src tests/

Level 2: Detailed Test Documentation

Detailed specifications for each test:

## Test File: test_user_service.py

**Path:** `tests/unit/test_user_service.py`
**Tests:** 15
**Coverage:** 95%

---

### Class: TestUserRegistration

Tests for user registration functionality.

---

#### Test: test_user_registration_with_valid_data

**Type:** Unit Test
**Line:** 45
**Status:** ✅ Passing
**Tags:** user, authentication

**Purpose:**
Verify that a new user can successfully register with valid email, username, and password.

**Preconditions:**
- Database is empty (no existing users)
- Email validation service is mocked

**Test Data:**

user_data = { "email": "test@example.com", "username": "testuser", "password": "SecurePass123!" }

**Test Steps:**
1. Create user data dictionary with valid fields
2. Call `UserService.register(user_data)`
3. Verify user record is created in database
4. Verify user ID is returned
5. Verify password is hashed (not plain text)

**Assertions:**
- `assert result.success == True`
- `assert result.user_id > 0`
- `assert User.query.count() == 1`
- `assert created_user.password != "SecurePass123!"`

**Expected Result:**
- User created with status "active"
- User ID returned (positive integer)
- Password stored as hash
- No errors raised

**Actual Result:** ✅ Pass (0.12s)

**Related Tests:**
- `test_user_registration_with_duplicate_email` - Tests duplicate handling
- `test_user_registration_with_weak_password` - Tests password validation

**Source Code Tested:**
`src/user_service.py:67-89` - `UserService.register()`

---

#### Test: test_user_registration_with_duplicate_email

**Type:** Unit Test
**Line:** 78
**Status:** ✅ Passing

**Purpose:**
Verify that registration fails when email already exists.

**Preconditions:**
- User with email "test@example.com" already exists in database

**Test Data:**

existing_user = User(email="test@example.com", username="existing") duplicate_data = { "email": "test@example.com", # Duplicate "username": "newuser", "password": "ValidPass123" }

**Expected Result:**
- `DuplicateEmailError` exception raised
- Error message: "Email already registered"
- No new user record created

**Actual Result:** ✅ Pass (0.08s)

---

### Class: TestUserAuthentication

Tests for user login and authentication.

[Continue with more tests...]

5. Include Coverage Analysis

Identify tested and untested areas.

## Test Coverage Analysis

**Overall Coverage:** 87%

### High Coverage Areas

**user_service.py** - 95% coverage
- ✅ User registration (all paths tested)
- ✅ User update (all paths tested)
- ✅ User deletion (all paths tested)
- ⚠️ Uncovered: External API error handling (lines 145-150)

**Recommendation:** Add test case `test_user_registration_with_