testing-assistant

Testing Assistant Skill

Expert system for testing and validating the Claude Patent Creator.

FOR CLAUDE: Test scripts in scripts/ directory.

• Go directly to running appropriate test
• Run from project root
• Tests require active venv
• Only run diagnostics if tests fail

When to Use

Running test suites, validating new features, testing after changes, debugging failures, creating tests, setting up CI/CD, performance testing, E2E validation, regression testing.

Testing Pyramid

         /\
        /  \       E2E (Manual + Automated)
       /----\
      / API  \     Integration Tests
     /--------\
    /  Unit   \    Unit Tests
   /----------\

Strategy: More unit tests (fast, isolated), fewer integration (moderate), minimal E2E (slow).

Test Suite Overview

scripts/
+-- test_install.py          # Complete installation validation
+-- test_gpu.py              # GPU detection and CUDA
+-- test_bigquery.py         # BigQuery connection
+-- test_analyzers.py        # Claims, spec, formalities
+-- test_embedding_speed.py  # Performance benchmarks
+-- test_checkpoint.py       # Index checkpoint system

Quick Test:

python scripts/test_install.py

Manual Testing via Claude

Test MCP tools through Claude Code interface.

Quick Test Examples

1. MPEP Search: "Search MPEP for claim definiteness requirements"
2. Patent Search: "Search for patents about neural networks filed in 2024"
3. Claims Review: "Review these claims: [paste test claims]"
4. Full Review: "/full-review" (with test application)
5. Diagrams: "Create a flowchart for this process: [describe]"

Validation Checklist

[OK] MPEP search returns relevant results
[OK] BigQuery search finds patents
[OK] Claims analyzer identifies issues
[OK] Specification analyzer checks support
[OK] Formalities checker validates format
[OK] Diagrams generate successfully
[OK] Full review workflow completes
[OK] All MCP tools accessible
[OK] Error messages clear and helpful
[OK] Performance acceptable (<2s most ops)

Creating New Tests

Quick Start

# Unit test template
def test_basic_functionality():
    from mcp_server.your_module import YourClass
    instance = YourClass()
    result = instance.method("test input")
    assert result is not None
    print("[OK] test_basic_functionality passed")

Test Categories:

1. Basic functionality
2. Edge cases
3. Performance
4. Error handling

Performance Testing

Quick Benchmark

from mcp_server.mpep_search import MPEPIndex
import time

index = MPEPIndex()
index.search("test", top_k=5)  # Warm up

start = time.time()
result = index.search("claim definiteness", top_k=5)
duration = time.time() - start
print(f"Search took: {duration:.3f}s")

Performance Thresholds

| Operation | Threshold | Notes | |-----------|-----------|-------| | MPEP search (first) | <3s | Model loading | | MPEP search (subsequent) | <500ms | Cached models | | BigQuery search | <2s | Network dependent | | Claims analysis | <3s | 20 claims | | Spec analysis | <10s | 10 pages | | Diagram generation | <1s | SVG output |

Troubleshooting Test Failures

| Problem | Solution | |---------|----------| | Import errors | Activate venv, pip install -r requirements.txt | | GPU tests fail | Check nvidia-smi, reinstall PyTorch, or skip | | BigQuery fails | Re-auth: gcloud auth application-default login | | Index not found | Rebuild: patent-creator rebuild-index | | Too slow | Check GPU usage, first run slower, check system load |

Best Practices

1. Test after every change
2. Automated testing
3. Test pyramid (more unit, fewer E2E)
4. Fast tests (<5 min suite)
5. Isolated tests (no dependencies)
6. Clear assertions
7. Document tests
8. Version control tests
9. Regular execution (weekly)
10. Monitor performance

Quick Reference

Run All Tests

python scripts/test_install.py
python scripts/test_gpu.py
python scripts/test_bigquery.py
python scripts/test_analyzers.py
python scripts/test_embedding_speed.py

Regression Test

python scripts/test_install.py || exit 1
python scripts/test_bigquery.py || exit 1
python scripts/test_analyzers.py || exit 1
echo "[OK] All regression tests passed!"

Manual Checklist

□ Ask Claude to search MPEP
□ Ask Claude to search patents
□ Ask Claude to review claims
□ Run /full-review command
□ Generate a diagram
□ Verify all tools work
□ Check performance (<2s)