quality-assurance

Quality Assurance

Quality is not a phase. It is not something you test into a product at the end. It is a property that emerges from how you work — from design decisions, process discipline, feedback loops, and the organizational culture that either surfaces defects early or buries them until production. W. Edwards Deming spent his career demonstrating that quality and productivity are not trade-offs — improving quality improves productivity by reducing rework, reducing waste, and building trust in the process.

Margaret Hamilton's Apollo guidance software team embodied this principle. Their "priority displays" concept — designing the computer to recognize overload conditions and autonomously shed non-critical tasks — was not a testing strategy. It was a quality architecture. The system was designed so that certain classes of errors were structurally impossible, and those that remained were handled gracefully. The most powerful quality assurance is the error that cannot occur because the system will not permit it.

Agent affinity: deming (quality/process), hamilton (systems engineering, error prevention through design)

Concept IDs: bus-business-planning, bus-business-ethics, crit-problem-analysis, crit-evidence-evaluation

Quality Framework Overview

| # | Framework | Core idea | Scope | |---|---|---|---| | 1 | Deming's 14 Points | Management creates quality | Organizational | | 2 | PDCA Cycle | Iterative improvement | Process | | 3 | Six Sigma (DMAIC) | Statistical defect reduction | Process / product | | 4 | TQM | Everyone owns quality | Organizational | | 5 | V&V | Verification vs. validation | Product | | 6 | Testing pyramid | Layered confidence | Product | | 7 | Peer review | Human defect detection | Code / design | | 8 | Kaizen | Continuous small improvements | Organizational |

Framework 1 — Deming's 14 Points for Management

Deming argued that 85% of quality problems are caused by the system, not by workers. Management creates the system; therefore, management is responsible for quality. His 14 Points:

1. Create constancy of purpose toward improvement of product and service.
2. Adopt the new philosophy. Management must lead the change.
3. Cease dependence on inspection to achieve quality. Build quality in.
4. End the practice of awarding business on price alone. Minimize total cost by working with fewer, trusted suppliers.
5. Improve constantly the system of production and service.
6. Institute training on the job.
7. Institute leadership. The aim of supervision should be to help people do a better job.
8. Drive out fear so everyone can work effectively.
9. Break down barriers between departments.
10. Eliminate slogans and targets that demand zero defects without providing methods.
11. Eliminate numerical quotas. Substitute leadership.
12. Remove barriers to pride of workmanship. Eliminate annual ratings.
13. Institute a vigorous program of education and self-improvement.
14. Put everybody to work to accomplish the transformation.

Application to Software

| Point | Software interpretation | |---|---| | 3. Cease dependence on inspection | Don't rely on QA to find bugs. Build quality through TDD, code review, and design. | | 5. Improve constantly | Retrospectives, refactoring, process experiments every sprint. | | 7. Institute leadership | Tech leads mentor; they don't command. | | 8. Drive out fear | Blameless postmortems. Psychological safety. | | 9. Break down barriers | Cross-functional teams. DevOps. No "throw it over the wall." | | 10. Eliminate slogans | "Zero bugs" as a goal without investing in prevention is a lie. | | 12. Remove barriers to pride | Let developers refactor. Let them fix the code that shames them. |

GSD connection. Deming's Point 3 — "build quality in" — maps directly to GSD's phase structure. The verify phase (gsd-verify-work) is not an inspection gate; it is a validation that the process itself produced the right outcome. If verify consistently fails, the problem is in discuss/plan/execute, not in verify.

Framework 2 — PDCA Cycle (Plan-Do-Check-Act)

Also called the Deming Cycle or Shewhart Cycle. The foundational improvement loop:

1. Plan. Identify an opportunity for improvement. Formulate a hypothesis: "If we do X, outcome Y will improve."
2. Do. Implement the change on a small scale. Collect data.
3. Check. Analyze the data. Did the change produce the expected improvement?
4. Act. If yes, standardize the change and expand it. If no, learn from the failure and return to Plan.

Worked Example — Reducing Code Review Turnaround

Problem. Code reviews take an average of 3 days, blocking merges and slowing velocity.

Plan. Hypothesis: "If we set a 24-hour SLA for first review response and limit PR size to 400 lines, review turnaround will drop below 1 day."

Do. Implement for 2 sprints. Track review turnaround time daily. Team agrees to prioritize reviews before starting new work.

Check. After 2 sprints: average turnaround dropped to 18 hours. PRs over 400 lines still take 2+ days. PRs under 400 lines average 8 hours.

Act. Standardize the 400-line limit. Add tooling to warn when PRs exceed it. Continue monitoring. Next cycle: investigate whether pair programming reduces review need.

PDCA and GSD

GSD's wave execution is a PDCA cycle at the phase level:

• Plan: PLAN.md creation (gsd-plan-phase)
• Do: Wave execution (gsd-execute-phase)
• Check: Verification (gsd-verify-work)
• Act: Retrospective and handoff (session report, handoff document)

Each session is a PDCA iteration. The carry-forward pattern (lessons from one session inform the next) is the "Act" step that closes the loop.

Framework 3 — Six Sigma (DMAIC)

Six Sigma aims for 3.4 defects per million opportunities. The DMAIC methodology is the improvement cycle:

1. Define. What is the problem? Who is the customer? What is the defect?
2. Measure. What is the current baseline? How do you measure the defect rate?
3. Analyze. What are the root causes? (5 Whys, fishbone diagram, Pareto analysis)
4. Improve. What changes address the root causes? Pilot and measure.
5. Control. How do you sustain the improvement? Monitoring, standard procedures, control charts.

Worked Example — Reducing Production Incidents

Define. Problem: 12 production incidents per month. Customer impact: 4.2 hours average downtime per incident. Goal: reduce to fewer than 4 incidents per month.

Measure. Baseline over 3 months:

• 36 total incidents
• Categories: deployment failures (15), database issues (8), third-party API failures (7), infrastructure (6)
• Mean time to resolution (MTTR): 2.1 hours

Analyze. Pareto chart shows deployment failures are 42% of incidents. Root cause analysis (5 Whys on the 3 worst deployment incidents):

Incident: Payment service crashed after deploy.

1. Why? New version had a null pointer exception.
2. Why? Code path not covered by tests.
3. Why? Unit tests did not cover the error branch.
4. Why? No test coverage requirement for deployments.
5. Why? CI pipeline does not enforce coverage thresholds.

Root cause: Missing automated quality gate in the deployment pipeline.

Improve.

• Add coverage threshold (80% minimum) to CI pipeline. Deployments blocked if not met.
• Add canary deployments: deploy to 5% of traffic, monitor for 15 minutes, then full rollout.
• Add automated rollback trigger: if error rate exceeds 1% in canary, auto-rollback.

Control.

• Monitor incident rate weekly. Alert if 3+ incidents in any week.
• Review canary rollback frequency monthly. High rollback rate means quality problems are being caught (good) but should also trigger upstream investigation (better).

Results after 3 months. Incidents dropped from 12/month to 3/month. Deployment-related incidents dropped from 5/month to 0.5/month. MTTR dropped from 2.1 hours to 0.8 hours (canary cat