gen-test-plan

Generate Test Plan

Analyze the repository's tech stack, branch changes vs default, and generate an executable YAML test plan focused on user-facing impact.

This is an E2E test plan — not an automated test wrapper. The generated plan will be executed by an autonomous agent acting exactly as a human QA tester would: launching real binaries, hitting real endpoints, interacting with real databases, and verifying real observable behavior.

Critical Rule: No Automated Test Duplication

NEVER generate test steps that re-run the project's existing automated test suite. This means:

• No cargo test, pytest, npm test, go test, mix test, or equivalent commands as test steps
• No wrapping unit/integration test modules in a test case
• No "run the tests and check they pass" — that's CI's job, not QA's

If you find yourself writing a test step that invokes the project's test runner, stop and rethink. Ask: "What would a human tester do to verify this feature works?" The answer is never "run the unit tests."

What E2E test steps look like:

• Build the binary and run it with real arguments, check stdout/stderr/exit code
• Start a server and hit it with curl
• Run a CLI command that writes to a real database, then query the database to verify
• Launch the TUI and verify it renders (via screenshot or process lifecycle)
• Chain multiple commands that exercise a full user workflow end-to-end

Hard gates

Complete these in order. Do not advance to the next gate until its Pass condition is met (each pass should leave retrievable evidence: pasted command output, a written list, or the generated file on disk). Scheduling: Gate 1 before Step 2; Gate 2 before Step 5; Gate 3 before Step 7; Gates 4–5 during Step 8 (after the Step 7 summary).

1. Diff and base pinned (after Step 1) — Resolve the base branch from --base when provided, otherwise use the repo default (main or master per Step 1). Compare HEAD to $(git merge-base HEAD origin/<base_branch>) (or equivalent if the remote ref differs). Pass: You record current_branch, base_branch, the merge-base SHA or range used, and changed_files from git diff --name-only <merge-base>..HEAD (empty list allowed if you paste or quote that output and state “no file changes vs base”).

1. Trace complete (after Step 4) — Pass: Every affected entry point you will test has a Core functionality vs Configuration/admin classification, and the Step 4 requirement holds: at least one test targets a core entry point or you document why that is impossible and flag manual review.

1. Plan file valid (after Step 6, before Step 7) — Pass: docs/testing/test-plan.yaml exists and the following command exits 0 (parses the YAML and asserts all four top-level keys are present — a single grep -E with alternations would pass on any one match, so do not substitute it):

``bash python3 -c "import sys, yaml; d = yaml.safe_load(open('docs/testing/test-plan.yaml')) or {}; missing = [k for k in ('version', 'metadata', 'setup', 'tests') if k not in d]; sys.exit('Missing keys: ' + ', '.join(missing) if missing else 0)" ``

1. No automated-test duplication (Step 8) — Pass: Every run: step and every services: command: is scanned for project test runners (cargo test, pytest, npm test, go test, mix test, jest, vitest, mocha, etc.); zero invocations. If any appear, remove or replace them with real E2E actions and re-run Gate 3.

1. Behavioral coverage (Step 8) — Pass: Re-read metadata.changes_summary and recent commit messages; at least one test’s context/steps exercises the primary user-visible behavior they describe. If they describe a capability (e.g., a new provider) but no step invokes it, add that test or fail verification.

Arguments

• --base <branch>: Base branch to diff against (default: main)
• Path: Target directory (default: current working directory)

Step 1: Gather Repository Context

# Get current branch
git rev-parse --abbrev-ref HEAD

# Resolve base branch: use --base if supplied, otherwise default (main → master)
BASE_BRANCH="${BASE_BRANCH:-$(git rev-parse --verify origin/main >/dev/null 2>&1 && echo main || echo master)}"
MERGE_BASE="$(git merge-base HEAD "origin/${BASE_BRANCH}")"

# Get changed files vs base
git diff --name-only "${MERGE_BASE}"..HEAD

# Get commit messages for context
git log --oneline "${MERGE_BASE}"..HEAD

Capture:

• current_branch: Branch name
• base_branch: Default branch to compare against
• changed_files: List of modified files
• commit_messages: What the PR is about

Step 2: Detect Tech Stack

See [references/stack-discovery.md](references/stack-discovery.md) for stack detection commands, entrypoint discovery, port discovery, and trace rules.

Step 3: Discover User-Facing Entry Points

A "user-facing entry point" is anything a human interacts with: CLI subcommands, HTTP endpoints, UI routes, TUI screens, gRPC services, database migrations, or configuration files that affect runtime behavior.

CLI Applications (Rust/clap, Python/argparse/click, Go/cobra)

# Rust (clap) — look for Subcommand derives and command enums
grep -rn "Subcommand\|#\[command\]" --include="*.rs" | head -20

# Python (click/typer/argparse)
grep -rn "@click.command\|@app.command\|add_parser\|add_subparser" --include="*.py" | head -20

# Go (cobra)
grep -rn "cobra.Command\|AddCommand" --include="*.go" | head -20

Build a map of:

• CLI subcommands: command name + description + file:line
• Required arguments and flags per subcommand
• Environment variables the binary reads (grep for env, std::env::var, os.Getenv, os.environ)

HTTP/API Services

Python (FastAPI/Flask):

grep -rn "@app\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | head -20
grep -rn "@router\.\(get\|post\|put\|delete\|patch\)" --include="*.py" | head -20

Node.js (Express/Fastify):

grep -rn "app\.\(get\|post\|put\|delete\)" --include="*.ts" --include="*.js" | head -20
grep -rn "router\.\(get\|post\|put\|delete\)" --include="*.ts" --include="*.js" | head -20

Rust (axum/actix/rocket):

grep -rn "Router::new\|\.route(\|#\[get\]\|#\[post\]\|HttpServer" --include="*.rs" | head -20

Go (net/http, gin, chi):

grep -rn "http.HandleFunc\|r.GET\|r.POST\|router.Get\|router.Post" --include="*.go" | head -20

Elixir (Phoenix):

grep -rn "get \"/\|post \"/\|pipe_through\|live \"/\|scope \"/\"" --include="*.ex" | head -20

Browser UI Routes

grep -rn "createBrowserRouter\|<Route\|path=" --include="*.tsx" --include="*.jsx" | head -20

Database and Migrations

# SQL migrations
ls migrations/ db/migrate/ priv/repo/migrations/ 2>/dev/null
# Schema files
ls schema.sql schema.prisma 2>/dev/null

Build a consolidated map of:

• CLI subcommands: name + args + file:line
• API endpoints: method + path + file:line
• UI routes: path + component + file:line
• Database migrations: filename + what they create/alter
• Configuration: env vars and config files that affect behavior

Step 4: Trace Changes to Entry Points

For each changed file, determine if it affects user-facing functionality:

1. Direct entry point change — File contains route definitions
2. Import chain analysis — Find what imports the changed file and trace up to entry points
3. Architecture-aware tracing — Read the project's CLAUDE.md, README, or architecture docs to understand data flow and module relationships, rather than relying solely on grep
4. Document the trace path in test context

Import Chain Analysis by Ecosystem

# Rust — use/mod/crate references and workspace deps
grep -rn "use.*<crate>\|mod <module>" --include="*.rs"
grep -rn "<crate-name>" --include="Cargo.toml"

# Python — from/import
grep -rn "from.*<module>\|import.*<module>" -