Work overview

Section 07 of 15

Govern

Artificial Intelligence Risk Management Framework (AI RMF 1.0)

National Institute of Standards and Technology · 2023

Contents

Section 07 of 15

  1. 01Executive Summary
  2. 02Framing Risk
  3. 03Audience
  4. 04AI Risks and Trustworthiness
  5. 05Effectiveness of the AI RMF
  6. 06AI RMF Core
  7. 07Govern
  8. 08Map
  9. 09Measure
  10. 10Manage
  11. 11AI RMF Profiles
  12. 12Appendix A: AI Actor Tasks
  13. 13Appendix B: AI Risks and Traditional Software Risks
  14. 14Appendix C: AI Risk Management and Human-AI Interaction
  15. 15Appendix D: Attributes of the AI RMF
Text size
Work overview

Section 7 of 15

Govern

National Institute of Standards and Technology · about 5 minutes

Original source ↗

5.1 Govern

The GOVERN function:

  • cultivates and implements a culture of risk management within organizations designing, developing, deploying, evaluating, or acquiring AI systems;

  • outlines processes, documents, and organizational schemes that anticipate, identify,

and manage the risks a system can pose, including to users and others across society – and procedures to achieve those outcomes;

  • incorporates processes to assess potential impacts;

  • provides a structure by which AI risk management functions can align with organizational principles, policies, and strategic priorities;

  • connects technical aspects of AI system design and development to organizational

values and principles, and enables organizational practices and competencies for the individuals involved in acquiring, training, deploying, and monitoring such systems; and

  • addresses full product lifecycle and associated processes, including legal and other

issues concerning use of third-party software or hardware systems and data.

GOVERN is a cross-cutting function that is infused throughout AI risk management and enables the other functions of the process. Aspects of GOVERN, especially those related to compliance or evaluation, should be integrated into each of the other functions. Attention to governance is a continual and intrinsic requirement for effective AI risk management over an AI system’s lifespan and the organization’s hierarchy.

Strong governance can drive and enhance internal practices and norms to facilitate organizational risk culture. Governing authorities can determine the overarching policies that direct an organization’s mission, goals, values, culture, and risk tolerance. Senior leadership sets the tone for risk management within an organization, and with it, organizational culture. Management aligns the technical aspects of AI risk management to policies and operations. Documentation can enhance transparency, improve human review processes, and bolster accountability in AI system teams.

After putting in place the structures, systems, processes, and teams described in the GOV- ERN function, organizations should benefit from a purpose-driven culture focused on risk understanding and management. It is incumbent on Framework users to continue to execute the GOVERN function as knowledge, cultures, and needs or expectations from AI actors evolve over time.

Practices related to governing AI risks are described in the NIST AI RMF Playbook. Table 1 lists the GOVERN function’s categories and subcategories.

Table 1: Categories and subcategories for the GOVERN function.

Categories Subcategories GOVERN 1: GOVERN 1.1: Legal and regulatory requirements involving AI Policies, processes, are understood, managed, and documented.

procedures, and 1.2: The characteristics of trustworthy AI are inte- GOVERN practices across the grated into organizational policies, processes, procedures, and organization related practices.

to the mapping, GOVERN 1.3: Processes, procedures, and practices are in place measuring, and to determine the needed level of risk management activities based managing of AI on the organization’s risk tolerance.

risks are in place, transparent, and GOVERN 1.4: The risk management process and its outcomes are implemented established through transparent policies, procedures, and other effectively. controls based on organizational risk priorities.

Continued on next page Table 1: Categories and subcategories for the GOVERN function. (Continued) Categories Subcategories GOVERN 1.5: Ongoing monitoring and periodic review of the risk management process and its outcomes are planned and organizational roles and responsibilities clearly defined, including determining the frequency of periodic review.

GOVERN 1.6: Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities.

GOVERN 1.7: Processes and procedures are in place for decommissioning and phasing out AI systems safely and in a manner that does not increase risks or decrease the organization’s trustworthiness.

GOVERN 2: GOVERN 2.1: Roles and responsibilities and lines of communi- Accountability cation related to mapping, measuring, and managing AI risks are structures are in documented and are clear to individuals and teams throughout place so that the the organization.

appropriate teams 2.2: The organization’s personnel and partners receive GOVERN and individuals are AI risk management training to enable them to perform their duempowered, ties and responsibilities consistent with related policies, proceresponsible, and dures, and agreements. trained for mapping, GOVERN 2.3: Executive leadership of the organization takes remeasuring, and sponsibility for decisions about risks associated with AI system managing AI risks.

development and deployment.

GOVERN 3: GOVERN 3.1: Decision-making related to mapping, measuring, Workforce diversity, and managing AI risks throughout the lifecycle is informed by a equity, inclusion, diverse team (e.g., diversity of demographics, disciplines, expeand accessibility rience, expertise, and backgrounds). processes are GOVERN 3.2: Policies and procedures are in place to define and prioritized in the differentiate roles and responsibilities for human-AI configuramapping, tions and oversight of AI systems.

measuring, and managing of AI risks throughout the lifecycle.

GOVERN 4: GOVERN 4.1: Organizational policies and practices are in place Organizational to foster a critical thinking and safety-first mindset in the design, teams are committed development, deployment, and uses of AI systems to minimize to a culture potential negative impacts.

Continued on next page Table 1: Categories and subcategories for the GOVERN function. (Continued) Categories Subcategories that considers and GOVERN 4.2: Organizational teams document the risks and pocommunicates AI tential impacts of the AI technology they design, develop, deploy, risk. evaluate, and use, and they communicate about the impacts more broadly.

GOVERN 4.3: Organizational practices are in place to enable AI testing, identification of incidents, and information sharing.

GOVERN 5: GOVERN 5.1: Organizational policies and practices are in place Processes are in to collect, consider, prioritize, and integrate feedback from those place for robust external to the team that developed or deployed the AI system engagement with regarding the potential individual and societal impacts related to relevant AI actors. AI risks.

GOVERN 5.2: Mechanisms are established to enable the team that developed or deployed AI systems to regularly incorporate adjudicated feedback from relevant AI actors into system design and implementation.

GOVERN 6: Policies GOVERN 6.1: Policies and procedures are in place that address and procedures are AI risks associated with third-party entities, including risks of inin place to address fringement of a third-party’s intellectual property or other rights.

AI risks and benefits 6.2: Contingency processes are in place to handle GOVERN arising from failures or incidents in third-party data or AI systems deemed to third-party software be high-risk.

and data and other supply chain issues.