Assessing societal impacts of AI systems

An AI impact assessment is a structured process to evaluate the potential consequences of developing or deploying an artificial intelligence system. It is a mandatory element of an AI governance risk and impact assessment to ensure systems are ethical, legally compliant, and safe for public use.

ISO/IEC 42001:2023 Annex A.5.5 mandates that organizations must assess and document the potential societal impacts of their AI systems continuously throughout the system's entire life cycle. This forms the baseline of an effective AI societal impact assessment.

Societal impact refers to broad consequences on communities, institutions, and the environment, such as effects on democracy, economic displacement, environmental sustainability, and public health. Learning how to assess societal impacts of AI systems requires looking beyond individual users to macroscopic outcomes.

The document should cover the intended use, foreseeable misuse, relevant demographic groups, environmental impacts, and proposed mitigations. Utilizing a standardized societal impact assessment template for AI ensures all necessary variables are documented for external auditors.

Organizations should perform a thorough stakeholder impact analysis for AI systems by consulting with domain experts, analyzing use cases, and identifying groups historically marginalized by similar technologies. Engaging directly with civil society and advocacy groups can also highlight unconsidered risks.

An AI risk assessment generally measures threats to the organization, such as financial loss or security breaches. Conversely, an algorithmic impact assessment specifically evaluates the negative consequences and harms experienced by external individuals, communities, and society at large.

The ISO 42001 impact assessment process requires evaluating impacts at the earliest design phase, right before formal deployment, and iteratively whenever there are significant updates to the model, its data sources, or its application context.

Organizations must review and update their AI harm and benefit assessment continuously as the system evolves or as real-world monitoring reveals unanticipated societal outcomes. Routine evaluations should also coincide with internal audit schedules.

Auditors look for completed impact assessment reports, documented mitigation plans, meeting minutes from cross-functional review boards, and clear evidence that societal impacts were weighed before greenlighting deployments to meet ISO 42001 requirements.

Yes, organizations can utilize frameworks like the NIST AI RMF impact assessment guidelines or tailor a responsible AI impact assessment template to capture system complexities. These templates help standardize criteria for measuring fairness, environmental impact, and societal well-being.

Societal impact assessments can become inconsistent when different teams use different templates, scoring scales, and approval paths. Tools like WatchDog Security's Compliance Center can help map required assessment steps to ISO/IEC 42001 Annex A.5.5, centralize evidence (impact reports, review minutes), and highlight gaps where assessments are missing or out of date.

An assessment is only useful if the identified harms translate into owned, tracked mitigation work with clear deadlines and outcomes. Tools like WatchDog Security's Risk Register can capture societal risks as formal risk entries, link them to treatment plans and approvals, and support executive reporting on residual risk and remediation status over time.