The Problem
An AI system that determines welfare eligibility, screens job applicants, or approves loans has real consequences for real people. Governments and regulators require structured assessments before deployment — not as paperwork, but as a decision-making tool that forces you to think through the consequences before they happen.
Two frameworks lead the field: Canada's Algorithmic Impact Assessment (AIA) and the EU AI Act's Fundamental Rights Impact Assessment (FRIA). Both require you to systematically evaluate the system's impact on rights, safety, and fairness before deployment.
Canada's AIA framework
Canada's AIA evaluates AI systems across multiple dimensions, scoring the system's impact level:
| Dimension | What It Measures | Scoring |
|---|
| Project context | Is the system automating a decision previously made by humans? | Higher impact if replacing human judgment |
| Decision impact | Does the system make decisions about people? | Higher impact for decisions about individuals |
| Affected populations | Who is affected? Vulnerable populations? | Higher impact for vulnerable groups |
| Impact scale | How many people are affected? | Higher impact for larger populations |
| Impact duration | How long do effects last? | Higher impact for irreversible effects |
| Reversibility | Can decisions be reversed? | Higher impact for irreversible decisions |
The final score determines the required governance level: from minimal reporting (low impact) to mandatory ministerial approval and public consultation (high impact).
EU AI act FRIA (article 27)
For high-risk AI systems deployed by public authorities or certain private entities, the EU AI Act requires a Fundamental Rights Impact Assessment:
- Intended purpose — What is the system supposed to do?
- Affected groups — Who are the data subjects?
- Specific risks — What fundamental rights could be violated?
- Mitigation measures — What's being done to address the risks?
- Human oversight — How will humans monitor the system?
- Complaint mechanism — How can affected individuals raise concerns?
Building the AIA tool
"""
algorithmic_impact_assessment.py
Structured algorithmic impact assessment based on Canada AIA and EU FRIA.
"""
import json
from datetime import datetime
AIA_DIMENSIONS = {
"decision_making": {
"name": "Decision Making Authority",
"question": "Does the system make or support decisions about people?",
"levels": [
{"score": 1, "label": "No decisions about people", "description": "System processes data but does not inform decisions about individuals"},
{"score": 2, "label": "Recommends decisions", "description": "System provides recommendations that humans review before acting"},
{"score": 3, "label": "Makes decisions with human override", "description": "System makes decisions but humans can override"},
{"score": 4, "label": "Fully automated decisions", "description": "System makes decisions without human review"},
],
},
"affected_population": {
"name": "Affected Population",
"question": "Who is affected by the system's outputs?",
"levels": [
{"score": 1, "label": "Internal staff only", "description": "Only employees are affected"},
{"score": 2, "label": "General public", "description": "General population, no specific vulnerable groups"},
{"score": 3, "label": "Specific demographic groups", "description": "Particular groups defined by age, gender, race, etc."},
{"score": 4, "label": "Vulnerable populations", "description": "Children, elderly, people with disabilities, marginalized groups"},
],
},
"impact_scale": {
"name": "Impact Scale",
"question": "How many people are affected?",
"levels": [
{"score": 1, "label": "< 100 people", "description": "Small scale, limited impact"},
{"score": 2, "label": "100-10,000 people", "description": "Medium scale"},
{"score": 3, "label": "10,000-100,000 people", "description": "Large scale"},
{"score": 4, "label": "> 100,000 people", "description": "Very large scale, population-level impact"},
],
},
"impact_duration": {
"name": "Impact Duration and Reversibility",
"question": "How long do effects last and can they be reversed?",
"levels": [
{"score": 1, "label": "Short-term, reversible", "description": "Effects are temporary and easily reversed"},
{"score": 2, "label": "Medium-term, reversible with effort", "description": "Effects last months, reversible with process"},
{"score": 3, "label": "Long-term, difficult to reverse", "description": "Effects last years, reversal requires significant effort"},
{"score": 4, "label": "Permanent, irreversible", "description": "Effects cannot be undone"},
],
},
"rights_impact": {
"name": "Fundamental Rights Impact",
"question": "Which fundamental rights could be affected?",
"levels": [
{"score": 1, "label": "No rights impact", "description": "System does not affect fundamental rights"},
{"score": 2, "label": "Privacy impact", "description": "System processes personal data"},
{"score": 3, "label": "Access to services", "description": "System affects access to credit, employment, housing, or education"},
{"score": 4, "label": "Liberty or safety", "description": "System affects freedom, physical safety, or legal rights"},
],
},
}
def conduct_aia(system_name, system_purpose, scores, mitigations=None):
"""
Conduct an algorithmic impact assessment.
Args:
system_name: Name of the AI system
system_purpose: Description of the system's purpose
scores: Dict mapping dimension_id to score (1-4)
mitigations: Optional list of mitigation measures
Returns:
Dict with AIA results and impact level
"""
if mitigations is None:
mitigations = []
results = {
"system_name": system_name,
"system_purpose": system_purpose,
"assessment_date": datetime.now().isoformat(),
"dimensions": {},
"total_score": 0,
"max_score": 0,
"impact_level": "",
"mitigations": mitigations,
"required_actions": [],
}
for dim_id, dim_data in AIA_DIMENSIONS.items():
score = scores.get(dim_id, 1)
level = next((l for l in dim_data["levels"] if l["score"] == score), dim_data["levels"][0])
results["dimensions"][dim_id] = {
"name": dim_data["name"],
"question": dim_data["question"],
"score": score,
"level_label": level["label"],
"level_description": level["description"],
}
results["total_score"] += score
results["max_score"] += 4
# Determine impact level
score_pct = results["total_score"] / results["max_score"]
if score_pct >= 0.75:
results["impact_level"] = "Very High"
results["required_actions"] = [
"Ministerial/leadership approval required",
"Public consultation recommended",
"Full FRIA required (EU AI Act Art. 27)",
"Mandatory human oversight with override capability",
"Post-deployment monitoring with monthly bias audits",
]
elif score_pct >= 0.5:
results["impact_level"] = "High"
results["required_actions"] = [
"Senior leadership approval required",
"FRIA recommended (EU AI Act Art. 27)",
"Human oversight with override capability",
"Post-deployment monitoring with quarterly bias audits",
]
elif score_pct >= 0.25:
results["impact_level"] = "Moderate"
results["required_actions"] = [
"Department head approval required",
"Human review of system outputs recommended",
"Annual bias audit",
]
else:
results["impact_level"] = "Low"
results["required_actions"] = [
"Standard project approval",
"Document system purpose and limitations",
]
return results
def generate_aia_report(aia):
"""Generate a Markdown AIA report."""
lines = []
lines.append(f"# Algorithmic Impact Assessment — {aia['system_name']}\n")
lines.append(f"**Purpose:** {aia['system_purpose']}")
lines.append(f"**Date:** {aia['assessment_date']}")
lines.append(f"**Impact Level:** {aia['impact_level']}")
lines.append(f"**Score:** {aia['total_score']}/{aia['max_score']}\n")
lines.append("## Assessment Dimensions\n")
lines.append("| Dimension | Score | Level | Description |")
lines.append("|---|---|---|---|")
for dim_id, dim in aia["dimensions"].items():
lines.append(f"| {dim['name']} | {dim['score']}/4 | {dim['level_label']} | {dim['level_description']} |")
lines.append("")
if aia["mitigations"]:
lines.append("## Mitigation Measures\n")
for m in aia["mitigations"]:
lines.append(f"- {m}")
lines.append("")
lines.append("## Required Actions\n")
for action in aia["required_actions"]:
lines.append(f"- [ ] {action}")
lines.append("")
lines.append("## Framework Mapping\n")
lines.append("| Framework | Requirement | Status |")
lines.append("|---|---|---|")
lines.append("| Canada AIA | Impact assessment completed | ✅ |")
lines.append("| EU AI Act Art. 27 | FRIA (if high-risk) | " + ("✅ Required" if aia["impact_level"] in ["High", "Very High"] else "Not required") + " |")
lines.append("| NIST AI RMF | MAP 1.6 — Characterize system impact | ✅ |")
lines.append("| ISO 42001 | Clause 6.1 — Risk identification | ✅ |")
return "\n".join(lines)
if __name__ == "__main__":
# Example: AIA for a loan approval system
aia = conduct_aia(
system_name="LoanApproval-GBM-v1",
system_purpose="Automated loan approval for consumer credit applications",
scores={
"decision_making": 3, # Makes decisions with human override
"affected_population": 3, # Specific demographic groups
"impact_scale": 3, # 10,000-100,000 people
"impact_duration": 3, # Long-term, difficult to reverse
"rights_impact": 3, # Access to services (credit)
},
mitigations=[
"Bias audit conducted with Fairlearn (Phase 2)",
"SHAP explanations provided for every decision (Phase 3)",
"Privacy impact assessment completed (Phase 4)",
"Human override available for all automated decisions",
"Post-deployment monitoring planned (Lesson 5)",
],
)
report = generate_aia_report(aia)
with open("algorithmic_impact_assessment.md", "w") as f:
f.write(report)
with open("algorithmic_impact_assessment.json", "w") as f:
json.dump(aia, f, indent=2)
print(f"Impact level: {aia['impact_level']}")
print(f"Score: {aia['total_score']}/{aia['max_score']}")
print(f"Required actions: {len(aia['required_actions'])}")
print("\nReport saved to algorithmic_impact_assessment.md")
You're assessing a system that automatically screens resumes and ranks candidates for a company with 50,000 employees. The system affects job applicants (general public) and makes recommendations that recruiters review. What impact level would you expect, and what governance is required?
The key insight is that "recommends" doesn't mean "low impact" — studies show that when an AI system ranks candidates, recruiters overwhelmingly follow the ranking (automation bias). The system effectively makes decisions even though it technically only recommends. The AIA captures this by scoring decision-making authority at 3 (makes decisions with human override) rather than 2 (recommends). The affected population is the general public (job applicants), the scale is large (50,000+ employees means many applicants), and the rights impact is access to employment. This puts the system in the High impact tier, requiring senior leadership approval, human oversight, and regular bias audits. Under the EU AI Act, this system is explicitly listed in Annex III as high-risk (employment decisions), so full compliance is mandatory.