Deployment Strategies — Rollback, Risk, and Validation
For deployment strategy patterns (canary, blue-green, rolling, progressive delivery), see cicd-and-deployment skill. This skill focuses on operational concerns: validation, rollback, risk, and post-deployment.
Pre-Deployment Validation
Before any deployment, validate:
- Deployment scripts tested in staging | Database migrations tested with rollback scripts
- Configuration management consistent across environments | Health checks and service discovery configured
- Monitoring and alerting systems prepared | Backup and disaster recovery procedures validated
Rollback Procedures
Design Rollback First
Every deployment plan starts with the rollback section:
- Database rollback: migration revert scripts tested and verified
- Application rollback: previous version tagged and deployable
- Configuration rollback: previous config snapshots available
- Traffic rollback: load balancer / feature flag kill switch ready
- Data rollback: data consistency plan for partial rollback scenarios
Automated Rollback Triggers
Configure automatic rollback when:
- Error rate exceeds baseline by >2x | P95 latency exceeds SLA threshold
- Health check failures exceed threshold (e.g., 3 consecutive) | Business metric anomaly detected (e.g., conversion drop >10%)
Manual Rollback Decision Criteria
Stakeholder-reported functional issues | Security vulnerability discovered post-deploy | Data integrity concerns | Performance degradation below acceptable levels.
Risk Assessment
Technical Risks
Integration failure with downstream services | Performance degradation under production load | Data migration integrity issues | Security vulnerabilities introduced.
Business Risks
User adoption challenges | Business process disruption | Stakeholder expectation misalignment.
Operational Risks
Infrastructure capacity limitations | Third-party dependency failures | Team availability for incident response.
Risk Mitigation Checklist
- Rollback procedure designed and tested
- Monitoring dashboards configured for deployment
- On-call team notified of deployment window
- Feature flags configured for gradual rollout
- Communication plan for stakeholders prepared
- Incident response runbook updated
Post-Deployment Validation
Production Smoke Tests
Critical user paths validated | Integration points tested with real external systems | Performance validated under production load | Security controls verified | Data integrity confirmed.
Monitoring Validation
Application performance metrics collecting | Error tracking and alerting active | Business metric dashboards updated | Infrastructure monitoring nominal.