The details of what amounts to enterprise AI governance best practices depend on the specific circumstances, including industry and risk profiles. A large corporation with multiple active AI use cases and autonomous or agentic AI systems naturally requires a different approach to responsible AI governance than a smaller or less AI-involved business. Still, the following principles may form part of an enterprise AI governance framework or AI governance policy for many types of organizations.
AI Laws and Regulations
As a starting point, corporate AI governance should be consistent with applicable AI regulations, both domestic and international. For instance, before it was abandoned, Canada’s draft Artificial Intelligence and Data Protection Act (AIDA) contained various obligations for AI systems, including pertaining to public information, risk assessment and mitigation, accountability frameworks, human oversight, and record keeping.
The European Union’s AI Act already sets forth requirements for fundamental rights impact assessments, conformity assessments, certification, and registration systems, which may be relevant for businesses with sufficient connections to the EU. In the US, a number of States – including Colorado and California – have also implemented AI-legislation, and draft legislation has been introduced at the federal level.
In addition to laws that specifically address AI, there are others with potential impacts on AI governance best practices that may have to be taken into account. These include the fields of privacy, data protection, intellectual property, torts, product liability, consumer protection, labour, competition, and platform regulation, among others.
AI Literacy
AI literacy at all levels of an organization is key. Leadership and decision-making bodies should ensure that, as a collective, they possess a level of understanding of AI that allows them to ask the right questions, make informed decisions, and exercise adequate oversight of AI. This can be achieved in different ways, including through individuals with in-depth AI knowledge, dedicated AI governance or ethics committees, and Chief AI Officers (CAIOs). It is also possible to delegate AI-related responsibilities to existing roles or bodies, such as governance committees, risk committees, and Chief Information Officers (CIOs).
Ideally, decision-makers should personally experience their organization’s use of AI tools, even if just as part of periodic hands-on demonstrations.
AI Governance Policies
Businesses need to implement and regularly update a comprehensive enterprise AI governance policy or suite of AI governance policies that address the procurement and use of AI throughout the business, including permitted uses and tools, risk profiles and risk assessments, transparency and explainability, record keeping, and risk management or internal controls.
Approaches in this regard may differ, ranging from an overarching general AI policy to a multi-layered approach with several connected AI policies and sub-policies, including for instance policies that address specific types of AI (such as agentic AI governance), specific applications (for example administrative uses, HR, credit decision, customer service), or specific types of user groups (finance, engineers, customers, business partners, etc.) within and outside the business. The higher the risk associated with these activities and applications, the more in-depth and detailed will policies addressing them tend to be.
In developing AI governance best practices and policies, organizations can now look to several national and international AI frameworks. A growing body of soft law on various aspects of corporate use of AI has emerged in recent years, ranging from general and sectoral guidance to self-regulatory approaches and uniform standards.
Among many others, these include:
- Canada’s Voluntary Code of Conduct on Responsible Development and Management of Advanced Generative AI Systems
- The National Institute of Standards and Technology’s (NIST) Artificial Intelligence Risk Management Framework
- The Office of the Superintendent of Financial Institutions’ Guideline E-23 on Model Risk Management
- Numerous AI-related rules and guidance for specific industries and professionals
- The ISO 42001 AI Management System Standard (ISO/IEC 42001:2023)
- The Institute of Electrical and Electronics Engineers’ (IEEE) various standards on autonomous and intelligent systems
Additionally, businesses and organizations can look to AI-related policies adopted by governments and NGOs, including Canada’s Directive on Automated Decision-Making and Ontario’s Responsible use of Artificial Intelligence Directive. The latter is similar to principles developed originally in the more broadly applicable (including to the private sector) European Union Ethics Guidelines for Trustworthy AI or the OECD’s AI Principles. While these are not directly applicable, they provide useful themes and approaches that business organizations may wish to incorporate in some forms in their own AI governance frameworks.
AI Awareness
Awareness that information shared with AI may be transferred to the system’s provider and/or used to train the AI is also important in order to maintain security and confidentiality. Among other potential issues, shared information may be discoverable in legal proceedings by regulators and private parties or subject to disclosure requests. For these reasons, confidential or sensitive information should only be shared with AI if necessary. Sharing should also be limited to the AI systems, scope, and purposes that are approved by the user’s organization.
Sharing of information may concern interactions with chatbots, virtual assistants, and uploaded documents or queries. But there are also less obvious sources of potential information transfers to be mindful of, such as through conferencing systems, automated note takers, and recording or transcription tools. AI is increasingly embedded in a broad range of applications, such that users may at times even be unaware they are using AI for work-related tasks.
AI Dependencies and Resource Management
The use of third-party AI products and services is widely recognized as a potential source of significant risks. Since most companies do not develop and control AI systems themselves, their insights and influence over AI are limited, while dependencies are amplified. A similar challenge stems from the fact that AI models may depend on supply chains of data, algorithms, and computational resources, which adds complexity by introducing fourth- and fifth-party risks. Given the severity of these risks, they should be appropriately addressed in AI governance frameworks.
In general, business leadership should review and potentially adjust their priorities and objectives in light of AI. In particular, AI needs to be recognized as an emerging essential resource, which includes the necessity to develop strategies for securing continued access to it, based on reasonable terms that are acceptable to the business. Businesses should also avoid undue dependencies on specific providers of AI services.
As part of the above considerations, it is important to carefully review agreements pertaining to the provision of AI services. Business organizations should, to the extent possible, demand that they retain control over their data and that it is safeguarded. Arrangements with third-party providers may require prior due diligence. It may also be prudent to include specific contractual provisions on users’ rights pertaining to information, monitoring, and audits as well as service levels and redress mechanisms.
Human Involvement and Oversight
To ensure AI systems work in ways that facilitate human decision-making rather than frustrate it, AI needs to work with and – to some extent at least – under the supervision of humans. In the corporate context, managerial and board oversight of AI is often regarded as a key factor for responsible AI governance, as is the idea that AI must not displace human judgment.
However, the appropriate level and nature of human involvement can vary depending on the nature of the AI’s activities and outputs. Close human oversight, such as a ‘human-in-the-loop’ model that requires human review and approval of AI decisions before they are acted upon, may be appropriate in the case of systems engaged in activities where the costs of error are especially high, that engage individuals’ legal rights, or that operate in ways that make it especially difficult to predict their outputs.
In certain cases, however, it can be sufficient to adopt a ‘human-on-the-loop’ model, under which an AI system is allowed to run autonomously, subject to some elements of human intervention. These elements may include human involvement in: (a) the design cycle for new AI systems; (b) testing of AI systems to ensure that outputs are in line with expectations; (c) post-deployment monitoring to ensure outputs remain aligned with expectations.
Additionally, humans need to retain the ability to take an AI system offline until the cause of an error is discovered and resolved. The latter is particularly relevant for organizations deploying autonomous AI. Indeed, agentic AI governance requires additional safeguards, including labeling, audit trails, and kill-switch mechanisms.
Transparency and Explainability
AI models, particularly deep learning systems, introduce challenges for businesses due to their complexity and opacity. Some AI systems may function as opaque ‘black boxes’ with decision-making processes that are difficult to interpret. Although new methods for tracking the inner workings of AI models are emerging, this poses issues for regulatory compliance, especially in areas requiring documentation and traceability of decisions. These difficulties are amplified when third-party AI is involved, further reducing transparency.
Despite these challenges, full explainability in every case is neither necessary nor practical. There is often a trade-off between explainability and performance, and different AI applications require different transparency levels. Consequently, instead of insisting on full interpretability, AI governance may, where appropriate, aim for a combination of outcome-based oversight, procedural safeguards and standards, and independent verification.
Outcome-based oversight prioritizes the assessment of AI outputs rather than a system’s internal workings. In terms of procedural safeguards and standards, AI governance should ensure that minimum standards for fairness, accuracy, and robustness are met. This may include establishing explainability benchmarks tailored to different AI use cases. Finally, prudent AI governance may include independent audits, stress-testing exercises, and external certification, especially for AI applications deemed high-risk or high-impact, with the aim of assessing AI reliability, transparency, absence of bias, and compliance.
Fiduciary Duties and AI
Business leadership should be aware that AI implicates their fiduciary duties when they manage and oversee AI as well as when they use AI themselves for governance tasks. Risks are typically tied to excessive or otherwise careless use of AI, but we may also see emerging duties that require not only responsible or competent use of AI, but also demand the adoption of available AI tools. Presumably, such duties may apply where AI use has become commonplace and such use demonstrably leads to improved outcomes.
It is also worth noting that traditional insurance may not cover liability for claims or harm related to AI. Existing insurance policies should therefore be reviewed and gaps and uncertainties addressed. Going forward, some organizations may need to consider obtaining specialized AI insurance.
This Insight is based in part on my co-authored submission to the Canadian Securities Administrators (CSA).