AI Ethics Review Skill
This skill produces a structured ethical review of an AI or machine learning feature, model, or product. Output covers fairness, transparency, privacy, safety, accountability, and societal impact — with risk scoring, prioritised mitigations, and a checklist suitable for governance review or responsible AI documentation.
⚠️ This skill provides a structured framework for identifying and documenting ethical risks. It is not a substitute for legal advice, regulated algorithmic impact assessments, or specialist ethics review required in specific jurisdictions (e.g. EU AI Act, UK AI regulation).
Required Inputs
Ask the user for these if not provided:
- Feature or model name and what it does
- Who it affects — which users or people does the AI interact with, make decisions about, or collect data from?
- What decisions or outputs it produces — recommendations, predictions, classifications, generation, automation?
- Consequentiality — how significant are the AI's decisions? (low-stakes suggestions vs decisions that affect employment, credit, health, safety, etc.)
- Data used — what training data, user data, or third-party data is used?
- Human oversight — is there a human in the loop, and at what stage?
- Deployment context — who will use this and how? (internal tool / consumer-facing / automated pipeline)
Output Structure
AI Ethics Review: [Feature / Model Name]
Product / system: [Name and brief description] Review type: [Pre-deployment review / Post-deployment audit / Change review] Risk tier: [High / Medium / Low — based on consequentiality, scale, and affected population] Reviewer: [Name / Team] Date: [Date] Status: [Draft / Approved / Requires escalation]
1. Feature Summary
| What it does | [1–2 sentences — plain English description of the AI feature and its purpose] |
| Who uses it | [End users / internal teams / automated system] |
| Who is affected by its outputs | [May be different from who uses it — e.g. an AI hiring tool is used by HR but affects candidates] |
| Output type | [Recommendation / Classification / Prediction / Generation / Automation / Scoring] |
| Scale | [How many people affected per day/month?] |
| Consequentiality | [High: affects access to services, employment, credit, health, safety / Medium: influences decisions / Low: suggestions with easy override] |
| Human oversight level | [Full automation / Human review before action / Human can override after action / Advisory only] |
2. Risk Tier Assessment
| Factor | Score (1–3) | Rationale |
|---|---|---|
| Consequentiality (impact on individuals) | [1=low, 3=high] | [e.g. 3 — model output influences hiring decisions] |
| Scale (number of people affected) | [1=few, 3=many] | [e.g. 2 — internal tool used for ~500 candidates/year] |
| Reversibility (can harm be undone?) | [1=reversible, 3=irreversible] | [e.g. 2 — unfair rejection can be appealed but may not be caught] |
| Vulnerability of affected group | [1=general population, 3=protected or vulnerable group] | [e.g. 2 — includes protected characteristics in the decision context] |
| Transparency (do affected people know?) | [1=informed, 3=opaque] | [e.g. 3 — candidates are not told AI is used in screening] |
Composite risk tier: [High (12–15) / Medium (7–11) / Low (3–6)]
Risk tier implications:
- High: Mandatory senior ethics review, DPA/DPIA required, human-in-loop for all consequential decisions, ongoing monitoring required
- Medium: Ethics review recommended, document mitigations, quarterly monitoring
- Low: Standard review, document assumptions, annual review
3. Fairness & Bias
Does the AI treat people equitably across groups?
Protected characteristics relevant to this feature: [List applicable protected characteristics — age, gender, race/ethnicity, disability, religion, national origin, etc.]
| Risk | Analysis | Mitigation |
|---|---|---|
| Training data bias | [Does the training data reflect historical discrimination? e.g. hiring data that reflects past biases in who was hired] | [Audit training data for demographic representation / use debiasing techniques / document data lineage] |
| Proxy discrimination | [Could the model use a proxy for a protected characteristic? e.g. using postcode as a proxy for race] | [Identify proxy features / test for disparate impact using adversarial debiasing] |
| Differential performance | [Does the model perform differently across demographic groups? — e.g. lower accuracy for underrepresented groups] | [Disaggregate performance metrics by group / set minimum performance thresholds per group] |
| Feedback loops | [Does the model's output reinforce existing disparities? e.g. recommending content that keeps disadvantaged groups in lower-engagement patterns] | [Monitor outcome distributions over time / implement feedback loop detection] |
Fairness evaluation method: [What method will be used to measure fairness — statistical parity / equalised odds / individual fairness? Who is responsible for running it and how often?]
4. Transparency & Explainability
Can affected people understand how the AI makes decisions?
| Dimension | Current state | Required state | Gap |
|---|---|---|---|
| User disclosure | [Are users told they're interacting with AI?] | [Yes — required for trust and regulation] | [e.g. No disclosure on current UI] |
| Decision explanation | [Can the system explain why it reached a conclusion?] | [For high-stakes decisions: yes] | [e.g. Black-box model — no feature attribution available] |
| Right to know | [Can affected people ask how a decision was made?] | [Yes — required under GDPR Art. 22 for automated decisions] | [e.g. No process exists] |
| Confidence calibration | [Does the model express appropriate uncertainty?] | [Yes — overconfident models cause over-reliance] | [e.g. Model outputs binary label without confidence score] |
Explainability approach: [LIME / SHAP / rule-based surrogate / LLM-generated rationale / none — and why]
5. Privacy & Data
Is personal data used responsibly and lawfully?
| Risk | Analysis | Mitigation |
|---|---|---|
| Data minimisation | [Does the model use more personal data than necessary?] | [Audit input features — remove any that don't improve performance and involve unnecessary data collection] |
| Data retention | [How long is personal data retained for training and inference?] | [Define retention policy aligned to GDPR / CCPA / sector requirements] |
| Re-identification risk | [Could model outputs or training data be used to identify individuals?] | [Differential privacy / k-anonymity / output rate limiting] |
| Third-party data | [Is data from third parties used? Is it licensed for this use?] | [Audit data licensing / get legal sign-off on each third-party source] |
| Cross-border data transfer | [Is personal data transferred across jurisdictions?] | [Legal review — Standard Contractual Clauses or equivalent] |
DPIA required? [Yes / No / Uncertain — for High tier or whenever processing is likely to result in high risk to individuals under GDPR Art. 35]
6. Safety & Reliability
What happens when the AI gets it wrong?
| Failure mode | Likelihood | Impact | Mitigation |
|---|---|---|---|
| False positives | [H/M/L] | [e.g. Flagging a legitimate transaction as fraud — customer locked out] | [Set threshold conservatively; human review for edge cases] |
| False negatives | [H/M/L] | [e.g. Missing a real fraud case — financial loss] | [Monitor false negative rate; set minimum recall threshold] |
| Out-of-distribution inputs | [H/M/L] | [Model behaves unpredictably on inputs outside training distribution] | [Input validation; confidence thresholding — route uncertain inputs to human review] |
| Model degradation | [M] |