Research Execute Skill

Description

Executes the research code in src/ to generate result artifacts in results/. This is Phase 3.5 of the research pipeline, sitting between Implementation (Phase 3) and Testing & Visualization (Phase 4).

Reads execution commands deterministically from the YAML frontmatter of plan/research_plan.md — no keyword heuristics, no entry-point guessing. The full run command is defined once during planning and executed here.

Usage

/research-execute [path/to/output/dir]

Arguments

• $ARGUMENTS — Optional path to the research output directory. If not provided, uses the most recent outputs/*/ directory.

Instructions

Claude-Only Mode

When --claude-only is active, there are no Gemini/Codex calls in this skill. All steps are performed by Claude directly.

Step 0: Locate Research Plan & Check Prerequisites

1. Find the active research output directory (from $ARGUMENTS or most recent outputs/*/).
2. Read plan/research_plan.md and parse the YAML frontmatter:

   ---
   languages: ["rust", "python"]
   ecosystem: ["cargo", "uv"]
   execution_cmd: "bash run_all.sh"
   dry_run_cmd: "bash run_all.sh --dry-run"
   expected_outputs:
     - "results/metrics.csv"
     - "results/checkpoint.pt"
   estimated_runtime: "~30 minutes"
   ---
   

3. Prefer execution_manifest.json: Check if execution_manifest.json exists in the output directory root. If it does, read execution fields from this file instead of the YAML frontmatter:

   {
     "schema_version": "1.0.0",
     "languages": ["rust", "python"],
     "ecosystem": ["cargo", "uv"],
     "execution_cmd": "bash run_all.sh",
     "dry_run_cmd": "bash run_all.sh --dry-run",
     "expected_outputs": [
       {"path": "results/metrics.csv", "required": true},
       {"path": "results/checkpoint.pt", "required": false}
     ],
     "estimated_runtime": "~30 minutes"
   }
   

   If execution_manifest.json exists, it takes precedence over YAML frontmatter fields. If it does not exist, fall back to the YAML frontmatter (backward compatibility).
4. If the frontmatter is missing or has no execution_cmd: announce the problem to the user and ask them to provide the execution command manually. Do not guess. Suggest adding the frontmatter to research_plan.md following the schema above.
5. Verify src/ exists and contains at least one file.

Step 1: Early Exit — results/ Already Populated

Check if results/ already exists and contains at least one file that is not run_log.txt, pre_execution_status.json, or pre_execution_status.md (legacy):

Glob: results/**/*

Exclusion: Exclude results/.staging/ from the existence check. Files under .staging/ are incomplete and must not trigger the 'results already exist' early-exit path.

If populated:

• Staleness check: Compute SHA-256 hashes of all files in src/ and plan/research_plan.md. Compare against hashes stored in results/.source_hashes.json (if it exists).
  • If hashes match: results are current. Announce: "results/ already contains artifacts and source code is unchanged. Skipping re-execution."
  • If hashes differ or .source_hashes.json is missing: Announce: "results/ contains artifacts but source code has changed since they were generated. Re-execution recommended." Ask the user: "(a) Re-execute with current code, or (b) Keep existing results?"
  • If user chooses (b): proceed to the write below.
• Write results/pre_execution_status.json (if not already present) with the canonical EXISTING schema:

  {
    "state": "EXISTING",
    "error_class": null,
    "severity": null,
    "retryable": false,
    "downstream_allowed": true,
    "traceback_ref": null,
    "next_action": "proceed"
  }
  

• Proceed directly to Step 6 (Summary).

Step 2: Dry-Run Verification

If dry_run_cmd is specified in the frontmatter, run it first as a fast sanity check:

{dry_run_cmd} 2>&1 | tee results/dry_run_log.txt

Timeout: 60 seconds.

On dry-run failure:

1. Classify the error:
   • Minor (wrong path, missing directory, missing results/ subdirectory, simple import): attempt one auto-fix, re-run dry-run.
   • Logic / Type / Shape error: report to user with the full traceback. Recommend rolling back to Phase 3 (Implement) to fix the code. Do NOT proceed to full run.
   • Environment / dependency error (missing binary, CUDA not found, missing package): report to user with install instructions. Do not attempt auto-fix.
   • Fatal / unknown (segfault, disk full, OOM, novel network error, or any error not matching the above categories): immediately write FAILED status to results/pre_execution_status.json. Do NOT attempt auto-fix. Report to user with full traceback and recommend investigating the root cause before retrying.
2. After auto-fix attempt: if dry-run succeeds → continue. If it still fails → stop and report.

If dry_run_cmd is not specified, skip this step and proceed directly to Step 3.

Step 3: User Checkpoint — Announce Full Run

Before executing the full run, announce:

Ready to execute:
  Command: {execution_cmd}
  Estimated runtime: {estimated_runtime or "unknown"}
  Output will be captured to: results/run_log.txt

Pause for user confirmation before running.

If estimated_runtime suggests a long job (> 15 minutes), add:

⚠ This job may take a long time. If you prefer to run it manually:
  1. Run externally: {execution_cmd}
  2. Copy results to the `results/` directory, then call `/research-execute [output_dir]` — the skill will detect existing results and skip re-execution automatically (Step 1 Early Exit).

Wait for explicit user confirmation before executing.

Step 4: Execute Full Run

Create results/ directory if it does not exist, then run:

Manifest overrides: If execution_manifest.json was loaded in Step 0:

• If cwd is specified, cd to that directory before executing the command.
• If env is specified (object of key-value pairs), prepend each as environment variable exports to the command (e.g., FOO=bar BAZ=qux {execution_cmd}).
• If timeout_override_ms is specified, use it instead of the default 30-minute timeout below.

Command validation: Before executing, inspect execution_cmd for shell metacharacters (;, &&, ||, |, $(, `, >, <, &). If any are found beyond simple pipes to tee, warn the user and require explicit confirmation before proceeding. Validate that the cwd field (if present) is a relative subdirectory path with no .. traversal and that the directory exists.

Execute in an isolated process group to prevent orphaned child processes on timeout:

setsid bash -c '{execution_cmd} 2>&1 | tee results/run_log.txt'

Timeout: 30 minutes (adjust based on estimated_runtime if provided and > 30 min, or timeout_override_ms from manifest).

On timeout — staggered teardown:

1. Send SIGTERM to the entire process group: kill -TERM -$PGID
2. Wait 10 seconds for graceful shutdown
3. If processes remain, send SIGKILL: kill -KILL -$PGID
4. Check for and release any lock files or temporary resources
5. Append "EXECUTION TIMED OUT. Process group terminated." to results/run_log.txt

Atomic results staging: To prevent half-written results from triggering false early-exit on subsequent runs:

1. Create staging directory: mkdir -p results/.staging/
2. Set environment variable RESULTS_DIR=results/.staging/ (if the execution script respects it) or configure output paths to write to results/.staging/
3. After execution completes successfully (exit code 0):
   • Move all files from results/.staging/ to results/: `m