ab-testing

A/B Testing

Test everything. Opinions are nice — data is better.

What to Test (Priority Order)

| Priority | Element | Expected Impact | |---|---|---| | 🔴 P0 | Headline | 10–50% lift | | 🔴 P0 | CTA text + color | 5–30% lift | | 🟡 P1 | Hero image/video | 5–20% lift | | 🟡 P1 | Form fields (fewer vs. more) | 10–40% lift | | 🟡 P1 | Social proof placement | 5–15% lift | | 🟢 P2 | Page layout (long vs. short) | 5–20% lift | | 🟢 P2 | Pricing display | 5–25% lift | | 🟢 P2 | Urgency messaging | 3–15% lift | | 🔵 P3 | Color scheme | 2–10% lift | | 🔵 P3 | Font choices | 1–5% lift |

Testing Rules

1. Test one variable at a time — Change only the element being tested
2. 50/50 split — Equal traffic to each variant
3. Minimum sample size — At least 100 conversions per variant before calling a winner
4. Statistical significance — Wait for 95% confidence before declaring a winner
5. Run for at least 7 days — Captures day-of-week variations
6. Document everything — Record hypothesis, variant details, and results

Test Hypothesis Template

HYPOTHESIS: If we change [element] from [current] to [proposed],
then [metric] will [increase/decrease] by [estimated %]
because [reasoning based on conversion principles].

TEST SETUP:
- Control (A): [Current version description]
- Variant (B): [New version description]
- Primary metric: [Conversion rate / Click rate / etc.]
- Secondary metric: [Revenue / Engagement / etc.]
- Required sample: [Number] visitors per variant
- Estimated duration: [X] days at [Y] daily visitors

Common Tests by Page Type

Opt-In Page

• Headline: Problem-focused vs. Solution-focused
• CTA: "Get Free Access" vs. "Download Now" vs. "Send Me the Guide"
• Form: Email only vs. Name + Email
• Social proof: Subscriber count vs. Testimonial

Sales Page

• Long-form vs. Short-form copy
• Video sales letter vs. Text
• Testimonials at top vs. After offer
• Payment: One-time vs. Payment plan (default)

Pricing Page

• 2 plans vs. 3 plans
• Annual default vs. Monthly default
• Feature comparison table vs. Simple list
• "Most Popular" badge placement

Results Tracking

After each test, log:

TEST: [Test Name]
DATE: [Start] → [End]
TRAFFIC: [Total visitors] ([Per variant])
RESULTS:
  Control: [X]% conversion ([N] conversions)
  Variant: [Y]% conversion ([N] conversions)
WINNER: [Control/Variant]
LIFT: [+/- X]%
CONFIDENCE: [X]%
NEXT: [What to test next based on learnings]