salesforce-apex-test-runner-skill

salesforce-apex-test-runner-skill

T1 read-only runtime skill for Apex test execution against a connected sandbox org. This skill runs tests, reads coverage, and diagnoses failures — it does not write code, deploy metadata, or touch production orgs. The View All Data permission is required by sf apex run test per Salesforce platform behavior; this permission must be granted only on the sandbox-only service account and NEVER on a production-eligible account.

When This Skill Owns the Task

Use salesforce-apex-test-runner-skill when the work requires live test execution:

• "Run the AccountServiceTest class in my sandbox"
• "Execute all local tests before deployment"
• "My test is failing with a governor limit error — run it and show me the stack trace"
• "Check code coverage for the Order trigger after my last change"
• "Run specific test methods: AccountServiceTest.testBulkSuccess"

Delegate elsewhere when:

| Situation | Skill to use | |---|---| | User needs test classes authored | salesforce-apex-test-generator-skill | | User needs production Apex authored | salesforce-apex-generator-skill | | Debug log analysis without running | salesforce-apex-log-analyzer-skill | | Static code review of test code | salesforce-apex-lwc-code-review-skill | | Sandbox deploy and validate | salesforce-deployment-validator-skill |

Required Context to Gather First

Before running any tests, confirm:

1. Target org alias — the --target-org value from sf org list. Never accept a raw

instance URL or session token.

1. Org type — must be sandbox. If production, stop immediately (see HARD REFUSAL).
2. Test scope — specific class names, specific method names, or test level

(RunLocalTests, RunAllTestsInOrg, RunSpecifiedTests).

1. Coverage threshold — default 75% per Salesforce deployment requirement; note if higher.
2. Goal — diagnosis only, or diagnosis + fix loop?

Recommended Workflow

Step 1 — Verify org alias and type

sf org display --target-org <alias>

Parse output for instanceUrl and check for sandbox indicators in the domain (.sandbox., .cs, .scratch., .scratch.salesforce.com). If instanceUrl contains login.salesforce.com or <companyname>.my.salesforce.com without sandbox indicators, treat as production and stop.

Step 2 — Confirm Connected App allowlist

The Connected App used by the Run As service account must have the target org alias explicitly in its allowlist. If the allowlist cannot be confirmed, stop and request the Connected App administrator verify the configuration.

Step 3 — Run the targeted test set

sf apex run test \
  --test-level RunSpecifiedTests \
  --tests <TestClassName1> <TestClassName2> \
  --target-org <alias> \
  --result-format json \
  --wait 10

For broader coverage checks:

sf apex run test \
  --test-level RunLocalTests \
  --target-org <alias> \
  --result-format json \
  --wait 30
  --code-coverage

Step 4 — Retrieve results if async

If the run was async (no --wait or timeout exceeded):

sf apex get test \
  --test-run-id <id> \
  --target-org <alias> \
  --result-format json \
  --code-coverage

Step 5 — Parse results

From the JSON output, extract:

• Pass count / fail count / skip count
• Overall code coverage percentage (target: >= 75%)
• Per-class coverage — identify classes below threshold
• Failing test methods — extract methodName, message, stackTrace
• Governor limit hits — scan message for "Limit" patterns

Step 6 — Diagnose failures

Consult references/failure-diagnosis.md for common patterns:

• DML in @TestSetup restrictions
• Governor limit errors in test context
• CalloutException — missing mock
• QueryException — data isolation issue (SeeAllData mis-use)
• Async result not visible — missing Test.startTest / Test.stopTest

Step 7 — Emit sanitized results with audit envelope

Apply redaction rules (see Redaction Rules section) before emitting any output. Emit the full audit envelope regardless of pass/fail outcome.

Step 8 — Recommend next steps

If failures found: recommend salesforce-apex-test-generator-skill for test corrections or salesforce-apex-log-analyzer-skill for deeper log analysis. If coverage below threshold: identify which classes are uncovered and recommend targeted test generation.

Quality Scoring Rubric (100-point)

Score the test run and analysis before presenting. Threshold: 80+ acceptable.

| Dimension | Points | What earns full marks | |---|---|---| | Test selection appropriateness | 25 | Scope matches the user's goal; RunSpecifiedTests used for targeted runs; RunLocalTests for pre-deploy coverage | | Sandbox-only enforcement | 25 | CRITICAL gate — org type verified as sandbox before any API call; HARD REFUSAL triggered for production targets | | Coverage analysis | 20 | Coverage percentage computed and compared to 75% threshold; per-class breakdown provided; uncovered classes named | | Failure diagnosis quality | 20 | Failures categorized by type; stack trace excerpted; remediation suggested from failure-diagnosis reference | | Audit envelope | 10 | All required audit fields present; timestamp accurate; org type verified field populated |

Scoring penalties:

• Sandbox check skipped: score voided (immediate HARD REFUSAL required)
• No coverage analysis when --code-coverage was available: -20
• Failures reported without diagnosis: -15
• Missing audit envelope: -15 (automatic caveat regardless of total)
• Org type not verified before execution: -25

T1 Least-Privilege Contract

This skill operates at T1 — read-only runtime (sandbox only).

• OAuth scopes: api and refresh_token only. No full, web, sfap_api, or other scopes.
• Run As account permissions:
• REQUIRED: View Setup and Configuration, View All Data (sandbox service account only)
• DENIED: ModifyAllData, ViewEncryptedData, ModifyMetadata, AuthorApex, ManageConnectedApps
• View All Data — important note: The sf apex run test command requires the View All Data

system permission for the running user per Salesforce platform behavior. This is why the Run As service account for this skill must be a sandbox-only account. This permission must NEVER be granted on an account used for production org access.

• No code mutation: This skill reads test results and coverage data. It does not write,

deploy, modify metadata, or create records outside of test context.

• Org allowlist: Connected App enforces which aliases are reachable. Skill verifies org

type before executing any sf apex command.

HARD REFUSAL — Production Org Targets

If the target org is identified as production, stop immediately and do not execute any sf apex command.

Production refusal response:

HARD REFUSAL: Apex test execution is not permitted on production orgs via this skill.

Reason: salesforce-apex-test-runner-skill operates at T1 sandbox-only scope. The
View All Data system permission required by sf apex run test must never be granted
on a production-eligible service account. Production Apex test runs also carry risk
of long-running test locks affecting live users.

Action required:
1. Identify a sandbox org alias for this test run.
2. Verify the Connected App allowlist includes the sandbox alias.
3. Re-invoke this skill with the sandbox alias.

If a production test run is genuinely required (e.g., for a post-deployment verification),
route through the human approval path via salesforce-live-guard-agent.

Audit Envelope Schema

Every execution emits an audit envelope. Emit even on HARD REFUSAL.

audit_envelope:
  matter_id: "<caller-provided-or-generated-uuid>"
  skill_id: "salesforce-apex-test-runner-skill"
  skill_version: "0.1.0"
  target_org_alias: "<alias>"
  run_as_user_id: "<user_id_placehol