This glossary defines 52 key terms as they are used in AI governance regulation, standards, and practice. Where a term has an authoritative definition in NIST, ISO, the EU AI Act, or the OECD, that source is cited. Where a term is used differently across frameworks, the most widely recognized usage is provided.
Adversarial Example An input to an AI system that has been deliberately crafted – often through barely perceptible modifications – to cause the system to produce an incorrect or harmful output. Adversarial examples exploit vulnerabilities in AI model design and are used in security research to test system robustness. See NIST AI RMF 1.0 (robustness discussion).
Agentic AI An AI system that can autonomously plan, make decisions, and take actions across software systems – including browsing the web, executing code, sending communications, or managing files – in pursuit of a goal, with minimal moment-to-moment human direction. Singapore’s IMDA Model AI Governance Framework for Agentic AI (May 2026) is among the first governance frameworks specifically designed for this category.
AI Literacy The knowledge, skills, and understanding that enable individuals to engage with AI systems effectively and critically – including understanding how AI works, its limitations, and its societal implications. EU AI Act Article 4 requires providers and deployers to ensure sufficient AI literacy among their staff as of February 2, 2025.
AI Management System (AIMS) The organizational system – comprising policies, processes, roles, controls, and oversight structures – used to manage AI risks and ensure responsible AI use throughout the AI lifecycle. ISO/IEC 42001:2023 is the internationally certifiable standard for AI management systems.
AI Red Team A group tasked with adversarially testing an AI system – attempting to identify vulnerabilities, elicit harmful outputs, or circumvent safety measures – before or after deployment. Red teaming is referenced in the ISED Voluntary Code of Conduct and the NIST AI 600-1 GenAI Profile as a core safety practice.
Algorithmic Impact Assessment (AIA) A structured evaluation process for assessing the potential impacts of an automated decision system on individuals and groups before deployment. Canada’s Treasury Board Directive on Automated Decision-Making requires federal departments to complete and publish an AIA before deploying any automated decision system. ISO/IEC 42005:2025 provides an internationally recognized standard methodology for AI system impact assessment.
Automated Decision-Making (ADM) The use of software or algorithms – including AI – to make or substantially contribute to decisions about individuals without meaningful human involvement in each individual decision. ADM triggering transparency and explanation obligations under Quebec Law 25, the Treasury Board Directive, and the EU AI Act’s provisions on automated profiling and decision-making.
Bias Audit An independent assessment of an AI system’s outputs to identify disparate impacts on protected groups. NYC Local Law 144 requires annual bias audits of automated employment decision tools by independent auditors, with public disclosure of results.
Confabulation See Hallucination. The term confabulation is preferred in some technical contexts because it more accurately describes the AI system’s behavior: generating plausible-sounding but factually incorrect content without awareness that it is doing so, rather than “hallucinating” in the human psychological sense. Used in NIST AI 600-1.
Conformity Assessment A process by which an AI system, organization, or AI management system is evaluated against defined standards or requirements – typically resulting in a certification, declaration, or attestation of conformity. The EU AI Act requires conformity assessments for high-risk AI systems before they can be placed on the EU market. ISO/IEC 42006:2025 establishes requirements for bodies performing AI management system certification.
Data Provenance The documented history of a dataset – where data originated, how it was collected, how it has been transformed, and who has had access to it. Data provenance is a core requirement under OSFI Guideline E-23, which requires that data used to develop models be “traceable and timely.”
Deployer An organization or individual that places an AI system into use for a specific purpose, operating within a context defined by the system’s original developer. Under the EU AI Act, deployers of high-risk AI systems have specific obligations including human oversight, monitoring, and post-market surveillance. Distinct from Provider.
Differential Privacy A mathematical privacy-preserving technique that adds calibrated statistical noise to datasets or model outputs, such that it becomes impossible to determine with high confidence whether any specific individual’s data was included in a dataset. Used in AI model training to protect individual privacy while preserving aggregate utility.
Drift The gradual degradation of an AI model’s performance over time, caused by changes in the real-world data patterns the model was trained on (data drift) or changes in the relationship between model inputs and outputs (concept drift). OSFI Guideline E-23 requires FRFIs to define monitoring standards that detect performance issues and breaches – drift detection is a core component.
Explainability The degree to which an AI system’s outputs, predictions, or decisions can be described in terms that humans can understand – including identifying which inputs most influenced the output. Explainability is a trustworthiness characteristic in NIST AI RMF 1.0 and an explicit requirement for high-risk AI systems under the EU AI Act.
Fairness The extent to which an AI system’s outputs treat individuals and groups equitably, without unjustified disparate impact on protected characteristics. NIST AI RMF 1.0 identifies fairness with harmful bias managed as a core trustworthiness characteristic. No single mathematical definition of fairness is universally applicable – choices between fairness metrics involve value judgments.
Federated Learning A machine learning approach in which a model is trained across multiple decentralized devices or servers holding local data, without the data being transferred to a central location. Federated learning can reduce privacy risks by keeping personal data on the originating device.
Fine-tuning The process of taking a pre-trained AI model – typically a large foundation model – and training it further on a smaller, domain-specific dataset to adapt its outputs for a particular task or context. Fine-tuning is a common approach for customizing general-purpose AI models for specialized applications.
Foundation Model A large AI model trained on broad data at scale, capable of being adapted to a wide range of tasks through fine-tuning or prompting. Examples include GPT-4, Claude, and Llama. The EU AI Act regulates a related category called General-Purpose AI (GPAI) models. The NIST AI 600-1 GenAI Profile addresses governance challenges specific to foundation models.
General-Purpose AI (GPAI) An AI model that displays significant generality and is capable of competently performing a wide range of distinct tasks, and which can be integrated into a variety of downstream systems or applications. The EU AI Act imposes specific obligations on GPAI model providers, including technical documentation, copyright compliance, and – for systemic-risk GPAI models – adversarial testing and incident reporting. GPAI obligations apply from August 2, 2025.
Generative AI AI systems capable of generating new content – text, images, audio, video, code, or synthetic data – in response to prompts. Generative AI includes large language models (LLMs) like ChatGPT and Claude, image generation systems like DALL-E and Midjourney, and multimodal systems. NIST AI 600-1 identifies unique risks specific to generative AI systems.
Governance The set of policies, processes, roles, accountability structures, and oversight mechanisms that determine how decisions about AI systems are made, who is responsible for them, and how compliance with internal standards and external regulations is ensured. In AI contexts, governance encompasses the full AI lifecycle – from design and procurement through deployment, monitoring, and decommissioning. See ISO/IEC 42001:2023.
Ground Truth The verified, accurate reference data against which an AI model’s predictions or outputs are evaluated. Models are typically trained and validated using labeled datasets where the ground truth is known. Garbage ground truth produces unreliable models – data quality and provenance are therefore core governance concerns.
Hallucination The generation by an AI system, particularly a generative AI model, of outputs that are factually incorrect, fabricated, or nonsensical, presented with apparent confidence. Hallucinations occur because generative AI models are optimized to produce plausible-sounding text, not verified facts. Treated as a primary risk in NIST AI 600-1 under the label “confabulation.” The Air Canada chatbot case (Chapter 5) is a documented real-world harm caused by AI hallucination.
High-Risk AI System An AI system whose use presents significant risks to health, safety, or fundamental rights, triggering heightened regulatory obligations. The EU AI Act defines high-risk AI systems through Annex III (covering credit scoring, employment, education, healthcare, law enforcement, and related domains). The Colorado AI Act uses a similar risk-based definition for its compliance obligations.
Human-in-the-Loop (HITL) A design approach in which a human is positioned to review, modify, or override an AI system’s output before it becomes a binding decision. HITL is a component of meaningful human oversight as required by EU AI Act Article 14. The degree to which HITL is “meaningful” depends on whether the human has sufficient information, training, time, and authority to actually exercise judgment – not merely approve outputs they cannot evaluate.
Impact Assessment See Algorithmic Impact Assessment. In AI governance more broadly, an impact assessment is any structured process for evaluating the potential positive and negative consequences of an AI system for individuals, groups, organizations, or society before or during deployment. ISO/IEC 42005:2025 provides a published international standard methodology.
Inference The process of using a trained AI model to generate outputs from new input data – i.e., the model’s operational use after training is complete. Inference is distinct from training. Most deployed AI systems operate in inference mode; governance of inference-stage behavior (monitoring, drift detection, output logging) is distinct from governance of the training process.
Interpretability The degree to which a human expert can understand and explain the mechanism by which an AI model produces its outputs – including the mathematical operations and the role of specific features or parameters in determining outputs. Interpretability is a stronger standard than explainability: an interpretable model can be examined at the level of its internal workings, whereas an explainable model need only produce outputs that can be explained in human terms. Both are referenced in NIST AI RMF 1.0.
Machine Learning (ML) A subset of AI in which systems learn patterns from data through statistical techniques, improving their performance on specific tasks without being explicitly programmed for each task. Supervised learning, unsupervised learning, and reinforcement learning are the primary categories. Most AI systems used in financial services, healthcare, and employment decisions are ML-based systems.
Model Card A structured document that accompanies an AI model and discloses its intended use, performance characteristics, training data, limitations, and ethical considerations. Model cards were first proposed by Google researchers in 2018 and have become a recognized transparency best practice endorsed by the IAPP and used across industry. The concept is now incorporated into many governance frameworks as a documentation requirement.
Model Risk Management (MRM) The discipline of identifying, assessing, and managing risks arising from the use of AI and other quantitative models in business decisions – including model error, misuse, and the consequences of model-driven decisions. MRM is the foundational governance framework for AI in financial services. OSFI Guideline E-23 establishes MRM requirements for all FRFIs, effective May 2027.
Multi-modal Describes an AI system that can process and generate more than one type of data modality – such as a system that accepts both text and images as inputs, or that can generate text, images, and audio. Multi-modal AI systems introduce governance challenges around the distinct risks associated with each modality.
Provider Under the EU AI Act, a provider is a natural or legal person, public authority, or other body that develops or has developed an AI system or a general-purpose AI model and places it on the market or puts it into service under its own name or trademark, whether for payment or free of charge. Providers have more extensive obligations than deployers under the EU AI Act framework.
Prompt Injection A cybersecurity attack against AI systems, particularly large language models, in which malicious instructions are embedded in inputs to the model – causing the model to ignore its original instructions and follow the attacker’s commands instead. Prompt injection is a significant security concern for any organization using AI systems that process external inputs.
RAG (Retrieval-Augmented Generation) A technique in which a generative AI system is combined with a retrieval mechanism – typically a database or document store – that provides relevant reference material to the model at inference time. RAG can reduce hallucination by grounding model outputs in retrieved documents, and is widely used in enterprise AI applications where accuracy and source attribution matter.
Red Teaming See AI Red Team. Red teaming refers to the practice of adversarially probing an AI system to identify vulnerabilities, harmful output modes, or failure cases before or after deployment. Distinguished from standard testing in that red teams actively seek to break the system, not merely verify that it works as intended.
Responsible AI A term used broadly to refer to AI development and deployment practices that are safe, fair, transparent, accountable, and aligned with human rights and societal values. No single universally binding definition exists, but responsible AI principles are articulated in the OECD AI Principles, NIST AI RMF 1.0, and ISO/IEC 42001:2023.
Risk Management The process of identifying, assessing, treating, monitoring, and communicating risks associated with AI systems throughout their lifecycle. NIST AI RMF 1.0 organizes AI risk management into four functions: Govern, Map, Measure, and Manage. ISO/IEC 23894:2023 provides guidance on integrating AI risk management into broader organizational risk management.
Robustness The ability of an AI system to maintain its intended performance and behavior across a range of operating conditions, including when inputs vary from what was seen in training, when data quality degrades, and when the system is subjected to adversarial attacks. Robustness is a trustworthiness characteristic in NIST AI RMF 1.0.
SaMD (Software as a Medical Device) Software that is intended to be used for one or more medical purposes – performing these purposes without being part of a hardware medical device. AI-enabled SaMD is regulated by the FDA under the AI/ML-Based SaMD framework and by Health Canada under the Medical Devices Regulations.
Sandbox A controlled testing environment in which new AI systems or features can be evaluated without affecting production systems or real users. Regulatory sandboxes allow organizations to test AI applications under regulatory supervision, with relaxed enforcement in exchange for transparency and data sharing. Several national regulators – including in the UK and Singapore – operate AI regulatory sandboxes.
Shadow AI The use of AI tools by employees in an organizational context without authorization, oversight, or knowledge of IT, legal, or compliance functions. Shadow AI is a significant governance risk: it means organizational data may be processed by AI systems that have not been reviewed, and it creates liability for AI-driven actions the organization cannot account for. The IBM Cost of a Data Breach Report 2025 found that shadow AI use added USD 670,000 to average breach costs for organizations with high levels of uncontrolled AI use.
Synthetic Data Data that is algorithmically generated rather than collected from real-world events or individuals, typically used to train or test AI models when real data is scarce, sensitive, or protected. While synthetic data can reduce privacy risks and enable model development in data-limited contexts, governance requires attention to whether synthetic data preserves the statistical properties – and biases – of the real-world data it is intended to represent.
Systemic Risk Risk that a failure in one component or institution could propagate through a larger interconnected system – in AI contexts, the risk that a widely used AI model or AI-enabled infrastructure, if it fails or is compromised, could cause cascading disruptions across financial systems, critical infrastructure, or other interdependent systems. The IMF’s May 2026 analysis of AI and financial stability explicitly identifies AI-enabled cyber threats as a systemic risk concern for the financial system.
Third-Party AI AI systems or components sourced from external vendors, partners, or open-source repositories, rather than developed internally. Third-party AI is covered by OSFI Guideline E-23, which requires FRFIs to apply the same model risk management standards to vendor-sourced AI as to internally developed models. The organization deploying third-party AI cannot transfer its governance obligations to the vendor.
Training Data The dataset used to train an AI model – the examples from which the model learns patterns. The quality, representativeness, and provenance of training data are foundational determinants of model performance and fairness. Biases present in training data tend to be reflected in model outputs. OSFI Guideline E-23 requires that data used to develop models be accurate, relevant and representative, compliant, traceable, and timely.
Transparency The degree to which information about an AI system – its capabilities and limitations, how it works, what data it uses, and what decisions it influences – is available to relevant parties, including affected individuals, regulators, and the public. Transparency is a foundational principle in the OECD AI Principles, the EU AI Act, and NIST AI RMF 1.0. Transparency requirements vary by risk level: high-risk systems require detailed technical documentation and individual-level explanations; limited-risk systems (chatbots) require disclosure that the user is interacting with AI.
Trustworthy AI AI that is designed and operated in accordance with a recognized set of properties that make it worthy of human and societal trust. NIST AI RMF 1.0 identifies the following as trustworthiness characteristics: validity and reliability, safety, security and resilience, accountability and transparency, explainability and interpretability, privacy-enhanced design, and fairness with harmful bias managed. The OECD AI Principles call for AI to be human-centred, trustworthy, transparent, explainable, robust, secure, and accountable.
Validation The process of evaluating whether an AI model performs as intended for its specified purpose, is fit for use in its deployment context, and meets defined performance standards. Validation is distinct from verification (which assesses whether the model was built correctly) – validation asks whether the right thing was built. Independent model validation – conducted by a function separate from model development – is required under OSFI Guideline E-23 and the U.S. Federal Reserve’s SR 11-7.
XAI (Explainable AI) A subfield of AI research and a set of methods and tools focused on making AI system outputs interpretable and explainable to humans. XAI methods include LIME (Local Interpretable Model-agnostic Explanations), SHAP (SHapley Additive exPlanations), and attention visualization. XAI is increasingly required for high-risk AI applications where affected individuals have a right to explanation, and for internal governance purposes where model owners must be able to account for model behavior to auditors and regulators.
