test

DLC: Test Coverage Analysis

Run the test suite, measure coverage, and create a GitHub issue for failures and coverage gaps.

Before running, read [../dlc/references/ISSUE-TEMPLATE.md](../dlc/references/ISSUE-TEMPLATE.md) now for the issue format, and read [../dlc/references/REPORT-FORMAT.md](../dlc/references/REPORT-FORMAT.md) now for the findings data structure.

Step 1: Detect Test Framework

Scan for test configuration:

| Config / Pattern | Framework | Coverage Tool | |-----------------|-----------|---------------| | vitest.config. / vite.config. (with test) | Vitest | @vitest/coverage-v8 | | jest.config.* / package.json (with jest) | Jest | Built-in (--coverage) | | pytest.ini / pyproject.toml (with pytest) / conftest.py | pytest | pytest-cov | | Cargo.toml + tests/ dir | cargo test | cargo-tarpaulin or cargo-llvm-cov | | go.mod + *_test.go files | go test | Built-in (-cover) | | .rspec / Gemfile (with rspec) | RSpec | simplecov |

Also detect test runner scripts in package.json or the project's task runner.

Step 2: Run Test Suite with Coverage

Execute tests with coverage enabled:

Select the tool based on availability (command -v) and detected framework, not exit codes — test runners exit non-zero when tests fail, which is a valid result to capture.

# Node.js (Vitest) — if vitest config detected
npx vitest run --coverage --reporter=json 2>/dev/null

# Node.js (Jest) — if jest config detected
npx jest --coverage --json 2>/dev/null

# Python
command -v pytest >/dev/null 2>&1 && pytest --cov=. --cov-report=json --tb=short 2>/dev/null

# Rust — select coverage tool by availability
if command -v cargo-tarpaulin >/dev/null 2>&1; then
  cargo tarpaulin --out json 2>/dev/null
elif command -v cargo-llvm-cov >/dev/null 2>&1; then
  cargo llvm-cov --json 2>/dev/null
fi

# Go
go test -cover -coverprofile=coverage.out -json ./... 2>/dev/null

# Custom test script — select runner by availability
if command -v bun >/dev/null 2>&1; then
  bun test 2>/dev/null
elif command -v npm >/dev/null 2>&1; then
  npm test 2>/dev/null
fi

Capture both:

• Test results: pass/fail status per test, failure messages
• Coverage metrics: line coverage %, branch coverage %, per-file breakdown

Step 3: Identify Coverage Gaps in Recent Changes

Focus on files changed recently (not just overall coverage):

# Files changed in last 5 commits
git diff --name-only HEAD~5..HEAD -- '*.ts' '*.js' '*.py' '*.go' '*.rs'

# Files changed vs main branch
git diff --name-only origin/main...HEAD

For each changed file:

1. Check if a corresponding test file exists
2. Check coverage percentage for that specific file
3. Flag files with < 80% coverage (or project-configured threshold)
4. Flag changed files with zero test coverage

TDD Commit Order Check

For each (source file, test file) pair identified above, check whether the test was written before the source — a signal of TDD discipline:

# Compute merge-base once — use $MB..HEAD consistently for both diff and log
MB=$(git merge-base origin/main HEAD)

# Get the commit hash and timestamp where the file was first added on this branch
git log $MB..HEAD --format="%H %ct" --diff-filter=A -- <source-file> | tail -1
git log $MB..HEAD --format="%H %ct" --diff-filter=A -- <test-file>   | tail -1

Scope: Only check files added on the current branch (the $MB..HEAD range above). Files that already exist in main are excluded — the TDD order signal is only meaningful for new files introduced in this branch. Reuse the same file list from git diff --name-only $MB..HEAD.

For each pair, first compare commit hashes, then timestamps:

| Condition | Action | |-----------|--------| | Either file has no --diff-filter=A output (shallow clone, renamed/moved file, or file not added on this branch) | Skip pair — log a note to stdout (not a finding): "Skipping TDD order check for {file}: insufficient git history" | | Both files share the same add-commit hash | No per-pair finding — same-commit pair counts as TDD-confirming for the branch-level check | | Add-commit hashes differ but timestamps are equal | Skip pair — log a note to stdout (not a finding): "Skipping TDD order check for {source} / {test}: unable to determine order (identical-second commit timestamps)" | | Source add-commit timestamp < test add-commit timestamp | Low tdd-order finding — test written after source (confirmation bias risk) | | Test add-commit timestamp < source add-commit timestamp | No finding — TDD discipline confirmed for this pair |

If at least one new source/test pair was evaluated, no new source file in the branch diff is missing a corresponding test file, and all evaluated pairs have pre-dating or same-commit tests, emit a single Info tdd-order finding: "TDD discipline confirmed for this branch."

Finding messages:

| Severity | Type | Message | |----------|------|---------| | Low | tdd-order | "Test for {source} written after source — risk of confirmation bias in test assertions" | | Info | tdd-order | "TDD discipline confirmed for this branch" |

Step 4: Classify Findings

Map results to the findings format from REPORT-FORMAT.md.

Severity mapping (reinforced here for defense-in-depth):

| Finding Type | Severity | |-------------|----------| | Test suite fails to run (config error, missing deps) | Critical | | Failing tests (regressions) | High | | Changed files with 0% coverage | High | | Overall coverage below project threshold | Medium | | Changed files with < 80% coverage | Medium | | Missing test file for new module | Medium | | Flaky tests (pass on retry) | Low | | Source file committed before its test file | Low | | Coverage informational metrics | Info | | TDD discipline confirmed (all tests pre-date source or are added in the same commit) | Info |

Step 5: Create GitHub Issue

Read [../dlc/references/ISSUE-TEMPLATE.md](../dlc/references/ISSUE-TEMPLATE.md) now and format the issue body exactly as specified.

Critical format rules (reinforced here):

• Title: [DLC] Testing: {summary — e.g. "3 failures, 72% coverage"}
• Label: dlc-test
• Body must contain: Scan Metadata table, Findings Summary table (severity x count), Findings Detail grouped by severity, Recommended Actions, Raw Output in collapsed details
• Include a Coverage Summary subsection in the metadata:

### Coverage Summary

| Metric | Value | Target |
|--------|-------|--------|
| Line Coverage | {n}% | 80% |
| Branch Coverage | {n}% | 70% |
| Files with 0% | {n} | 0 |

Acquire BRANCH for the Scan Metadata table above — ISSUE-TEMPLATE.md's lifecycle acquires REPO itself. Then follow ISSUE-TEMPLATE.md's Issue Creation Command lifecycle exactly — substitute {skill-name} = test, {Type} = Testing, {type} = test, {additional-required-sections} = '## Raw Output':

BRANCH=$(git branch --show-current)
echo "BRANCH=$BRANCH"   # ISSUE-TEMPLATE.md's Write step is a separate tool call and can't see this shell's variables

Step 6: Report

Test analysis complete.
  - Framework: {detected framework}
  - Tests: {passed} passed, {failed} failed, {skipped} skipped
  - Coverage: {line}% line, {branch}% branch
  - Changed files coverage: {n}/{total} above threshold
  - Issue: #{number} ({url})

If all tests pass and coverage is above threshold, skip issue creation and report: "All tests passing. Coverage meets threshold ({n}%)."