When to use this skill
Trigger when:
- Running active testing against a target with active SOC monitoring
- A confirmed-vulnerable finding stops reproducing on recheck
- Baseline timing shifts unexpectedly (3× slower, sudden errors, new headers)
- Response sizes change between test windows
- New WAF cookies or headers appear that weren't there at session start
- Lockout / error rates change between test windows (especially LOCKED count for credential attacks)
- Engagement is "assume breach" or "white box" — client knows you're testing
DO NOT use for:
- Bug bounty (client doesn't know you're there; no real-time IR)
- Pure recon (no state-change happening)
- One-off vulnerability scanning (no temporal dimension)
The core insight
In a real red-team engagement against a competent SOC, the security state of the target is not static. It changes during your test in response to your traffic. These state changes are:
- Themselves valuable findings (positive operational observations about IR responsiveness)
- Confirmation evidence (mid-engagement patch = the original vulnerability was real)
- Classification signals (WAF rule deployment vs code fix — different remediation depth)
Anti-pattern: treating reproduction failure as evidence the original signal was a false positive. Original PoC artifacts captured before the change are still the vulnerability finding.
The discipline — capture before, diff after
Before any active test:
# Capture pre-test fingerprint of the target
fingerprint = {
"ts_pre": time.time(),
"ip_seen": "<operator-src-ip>",
"baseline_response_time_ms": <measure>,
"baseline_response_size_bytes": <measure>,
"response_headers": <capture set>,
"waf_cookies": <list>,
"lockout_count_in_state": <count from o365_attempts.json>,
}
Persist to engagement_log/fingerprint_pre.json.
During the test:
Log every test result with full context (timestamp, IP, payload, response code, response size, response time, headers if relevant) to JSONL append-only.
After the test session OR on first failed-recheck:
fingerprint_post = same structure
delta = {
"baseline_time_change_ms": post.time - pre.time,
"baseline_size_change_bytes": post.size - pre.size,
"new_headers_appeared": post.headers - pre.headers,
"new_waf_cookies": post.cookies - pre.cookies,
"new_lockouts": post.locked_count - pre.locked_count,
}
If any delta is significant — investigate, don't retract.
The three primary IR observations
Observation 1 — Mid-engagement WAF rule deployment
Symptoms:
- Original payloads return identical response → no signal at all on recheck
- Body size identical to baseline (login page reflection)
- Timing reverts to baseline regardless of payload
- New cookie or header in responses (e.g.,
cf-bm,__cf_bm,awselb) - Specific keyword in URL/body now triggers different response code (403, 406, 429)
Confirmation: retry with WAF-evasion variants:
- URL-encode the payload differently (
%27vs%5cu0027) - Change request method (POST → PUT, GET → POST)
- Different content-type (form-urlencoded → multipart)
- Slower pace (5s → 60s between requests)
- Mixed-case keywords (
SLEEP→SlEeP)
If WAF-evasion variants restore the signal, the mitigation is at the WAF layer (bypassable).
If even WAF-evasion variants stay blocked, the mitigation is likely in code.
Finding template:
Subject: Mid-engagement mitigation deployed for <vulnerability X>
Observation: At engagement timestamp T0, vulnerability <X> on <endpoint> was
confirmed via <PoC>. At T0+<minutes>, recheck via the original payload no longer
reproduces the timing/error/size differential. <WAF-evasion variant> [does/does
not] restore the signal.
Description: This pattern is consistent with the client SOC observing engagement
traffic and deploying a mitigation in real time. Mitigation depth assessment:
[at-WAF, bypassable] vs [in-code, durable].
Impact (positive): Client SOC has both detection-grade visibility into application
traffic AND the authority to deploy mitigations within ~<minutes> of detection.
Impact (caveat): The original vulnerability did exist and was exploitable for at
least the engagement window before mitigation. If the mitigation is at the WAF
layer only, the underlying code-level flaw remains exploitable via alternative
payloads.
Recommendation:
1. Verify the mitigation is in code (parameterized queries, input sanitization),
not just at the WAF layer.
2. Audit the codebase for the same root cause across sister applications.
3. (Positive) Document the IR responsiveness as a capability metric.
Observation 2 — Active concurrent attacker
Symptoms:
- Many
AADSTS50053(LOCKED) responses despite your 1-attempt-per-user discipline - Lockouts cluster alphabetically or by some other sort key
- New lockouts appear DURING your engagement (diff before/after)
- LOCKED rate exceeds expected baseline (in our engagement: 11% of all attempts → red flag)
Math check:
- Your discipline: 1 attempt per user lifetime
- Smart Lockout default: 10 fails / 10 min
- Therefore: you cannot mathematically cause Smart Lockout
- Therefore: every AADSTS50053 you see was caused by someone else
Confirmation:
- Sort locked accounts alphabetically; if they cluster, attacker is using sorted username list
- Compare pre-session lockout count vs post-session — new locks during your session = attacker is active right now
- Probe a known-active "system" account (
noreply@,info@,oc@) — if it's locked, attacker is hitting service mailboxes too (typically MFA-exempt → high-value to attackers)
Finding template:
Subject: Active external password-spray campaign detected during engagement
Observation: During M365 ROPC validation against the <tenant> Entra tenant, <N>
unique principals returned AADSTS50053 (Smart Lockout) when probed with a single
password attempt at safe pace. With our hard cap of 1 attempt per user, we cannot
mathematically cause these lockouts. The locks are pre-existing and continue to
accumulate during our engagement window — <K> NEW locks observed between
<timestamp_start> and <timestamp_end>.
Description: The pattern (alphabetical clustering, real-time accumulation,
including system mailboxes) is consistent with an external attacker performing
a username-list-driven password spray attack against the tenant.
Impact: An external adversary is actively attempting to compromise corporate
M365 accounts. The attacker has knowledge of the user-email schema and a
password-guess wordlist. <List of locked accounts is now in attacker's hands>
(Smart Lockout differentiates valid from invalid usernames).
Recommendation (CRITICAL — within 24h):
1. Open a P1 incident with the SOC. Pull Entra sign-in logs for the <N> locked
accounts over the last 30-60 days. Identify source IPs and time windows.
2. Apply Conditional Access rule blocking sign-ins from outside <expected geo>
for non-admin accounts.
3. Enable Identity Protection's User Risk policy with auto-reset on high risk.
4. Force tenant-wide password reset for all <N> previously-locked accounts.
5. Audit service accounts for MFA exemptions; ensure all human-interactive
accounts have phishing-resistant MFA.
Evidence: engagement_log/poc/m365/locked_accounts.txt (<N> entries with timestamps)
Observation 3 — Detection-induced rate limiting / IP blocks
Symptoms:
- Specific IP starts returning 403 / 429 / 451 after a window of normal responses
- Specific IP starts seeing dramatically slower responses (3x+ baseline)
- TLS handshake fails or RST mid-connection
- DNS suddenly returns NXDOMAIN for hosts that resolved before
- Some hosts work from one IP but not another
Confirmation:
- Rotate to a different IP and retry — if works, you got rate-limited/blocked
- Compare TTL on DNS responses — sudden short TTL = active mitigation deployed
- Check `