claude-skills-vault

Testing Automation Expert

Master production-grade testing strategies for Python and TypeScript applications. QA Architect level - design test ecosystems that scale.

When to Use

• Designing test strategies for features/projects
• Setting up pytest, Jest, Vitest, or Playwright
• Improving test coverage and quality
• Contract testing, accessibility, mutation testing
• CI/CD test pipeline optimization
• Test architecture review or refactoring

Triggers

• /test-plan - Design test strategy for a feature/project
• /fixture-refactor - Optimize test fixture architecture
• /e2e-setup - Set up Playwright E2E testing
• /test-audit - Review test coverage and quality
• /contract-test - Set up Pact contract testing

Reference Files

Load the appropriate reference based on need:

| Category | Framework | Reference | |----------|-----------|-----------| | Python | pytest | references/python-pytest.md | | TypeScript | Jest | references/typescript-jest.md | | TypeScript | Vitest | references/typescript-vitest.md | | E2E | Playwright | references/e2e-playwright.md | | Advanced | Contract/A11y/Mutation | references/specialized-testing.md |

Test Quality Checklists

Pre-Implementation Checklist

• [ ] Test strategy defined - unit, integration, E2E, contract boundaries clear
• [ ] Fixtures planned - factory functions, DB setup, cleanup
• [ ] Mocking strategy - mock at boundaries only, not internals
• [ ] Coverage targets - minimum 80% branches/lines/functions
• [ ] CI integration - tests run on every PR with sharding if >10min
• [ ] Flakiness detection - pytest-randomly / random test order enabled

Unit Test Checklist

• [ ] Tests isolated - no shared state between tests
• [ ] One assertion focus - test one behavior per test
• [ ] Descriptive names - should_return_error_when_user_not_found
• [ ] AAA pattern - Arrange, Act, Assert clearly separated
• [ ] Edge cases covered - null, empty, boundary values
• [ ] No external dependencies - DB, network, filesystem mocked
• [ ] Property-based tests - Hypothesis/fast-check for edge case discovery

Integration Test Checklist

• [ ] Real dependencies - use Testcontainers (not SQLite for Postgres!)
• [ ] Transaction rollback - isolate tests with rollback
• [ ] API contract tested - request/response shapes verified
• [ ] Error paths tested - timeout, connection errors
• [ ] Auth flows tested - token generation, validation
• [ ] Declarative assertions - dirty-equals for complex JSON

E2E Test Checklist

• [ ] Critical paths covered - login, checkout, core flows
• [ ] Page Object Model - locators abstracted into classes
• [ ] Resilient selectors - role/label over CSS/xpath
• [ ] Network mocking - test error states without real API
• [ ] Visual regression - Docker for consistent screenshots
• [ ] Cross-browser - Chrome, Firefox, Safari, mobile
• [ ] Auth state reuse - save/load storage state
• [ ] Trace artifacts - uploaded for CI debugging

Architecture Checklist (Microservices)

• [ ] Contract tests - Pact CDCT between services
• [ ] API fuzzing - schemathesis against OpenAPI spec
• [ ] Mutation testing - mutmut/Stryker validates test quality
• [ ] Accessibility - axe-core in E2E pipeline
• [ ] Performance baselines - k6/Locust load tests
• [ ] CI sharding - parallel machines for >10min suites

Test Maintenance Checklist

• [ ] No flaky tests - deterministic, no sleep()
• [ ] Fast execution - unit <5s, integration <30s, E2E <2min
• [ ] Readable failures - clear error messages
• [ ] DRY fixtures - reusable factories, not copy-paste
• [ ] Regular cleanup - delete obsolete tests
• [ ] Quarantine flaky - .flaky.spec.ts until fixed

Testing Pyramid

         /\
        /E2E\        Few, slow, high confidence
       /-----\
      /Integ. \      Some, moderate speed
     /---------\
    / Contract  \    Service boundaries
   /-------------\
  /    Unit       \  Many, fast, isolated
 /------------------\

| Type | Count | Speed | Scope | |------|-------|-------|-------| | Unit | 60% | <100ms | Single function | | Contract | 10% | <1s | Service interface | | Integration | 20% | <5s | Service + DB | | E2E | 10% | <30s | Full user flow |

Test Organization

tests/
├── conftest.py / setup.ts   # Root fixtures
├── factories/               # Test data factories
│   └── user.factory.ts
├── fixtures/                # Static test data
│   └── users.json
├── unit/                    # Fast, isolated
│   ├── services/
│   └── utils/
├── integration/             # DB, external services
│   ├── api/
│   └── repositories/
├── contract/                # Pact consumer/provider
│   ├── consumer/
│   └── provider/
├── e2e/                     # Full user flows
│   ├── pages/               # Page objects
│   ├── specs/
│   └── a11y/                # Accessibility tests
└── load/                    # k6/Locust scripts
    └── api-load.js

Anti-Patterns

| Don't | Do | |-------|-----| | Test implementation details | Test behavior/contracts | | Hardcode test data | Use factories/fixtures | | Share state between tests | Isolate with fixtures | | Mock everything | Mock at boundaries only | | Use time.sleep() | Use wait_for or time mocking | | Use SQLite for Postgres tests | Use Testcontainers | | Write flaky time-based tests | Use time-machine / vi.useFakeTimers() | | Skip edge cases | Use property-based testing | | Ignore test failures | Fix or remove immediately | | One giant conftest.py | Split into modules | | Integration tests for service contracts | Use Pact contract testing | | Skip accessibility | Add axe-core to E2E | | Trust coverage alone | Add mutation testing |

Specialized Testing

Contract Testing (Microservices)

# Consumer test (Python)
pact = Consumer("UserService").has_pact_with(Provider("AuthService"))

See references/specialized-testing.md for full Pact patterns.

API Fuzzing (FastAPI/OpenAPI)

schemathesis run http://localhost:8000/openapi.json

Auto-generates tests from OpenAPI spec - finds edge cases.

Mutation Testing

# Python
mutmut run && mutmut results

# TypeScript
npx stryker run

Validates test quality - surviving mutants = gaps.

Accessibility Testing

const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);

Quick Reference

Python (pytest)

pytest                              # Run all
pytest -m "not slow"                # Skip slow
pytest --cov=src --cov-report=html  # Coverage
pytest -n auto                      # Parallel
pytest -x --lf                      # Stop first, re-run failed
pytest -p randomly                  # Randomize order (flakiness detection)

TypeScript (Vitest)

npx vitest                  # Watch mode
npx vitest run              # Run once
npx vitest --coverage       # Coverage
npx vitest --ui             # Browser UI

Playwright

npx playwright test                 # Run all
npx playwright test --shard=1/4     # CI sharding
npx playwright test --debug         # Debug mode
npx playwright show-trace trace.zip # View trace

Recommended Packages

Python

# Core
pip install pytest pytest-asyncio pytest-cov pytest-xdist
pip install pytest-mock pytest-randomly time-machine

# Advanced
pip install syrupy hypothesis dirty-equals
pip install testcontainers[postgres] httpx
pip install schemathesis pact-python mutmut
pip install playwright pytest-playwright axe-playwright-python
pip install locust  # Load testing

TypeScript

# Vitest (recommended for new projects)
npm install -D vitest @vitest/coverage-v8 @vitest/ui

# Jest (legacy/React Native)
npm inst