AI Governance: Managing AI Responsibly at Scale
Governance is a design function, not a review gate. See how voluntary frameworks, model documentation, and human-feedback economics shape it.
AI governance is usually presented as a stage that comes after the model is built: someone reviews the risks, writes a policy, and the organisation becomes "responsible". That sequence does not survive contact with a system that has to run in production at scale, because by the time a model is finished, most of the decisions that create risk have already been made — in what data was collected, what was never measured, and what the optimiser was rewarded for.
Direct answer:
Governing AI at scale means three things done continuously rather than once: producing documentation that makes a system's intended use and measured limits reviewable, deciding where human input goes and how much of it is needed, and adopting a shared measurement vocabulary so claims about AI systems can be compared across jurisdictions and sectors.
The frameworks that exist are mostly voluntary, which is precisely why the artefacts they produce — model cards, dataset datasheets, risk profiles — matter more than the frameworks themselves.

Voluntary frameworks, and why that shapes the work
The most widely referenced government framework for AI risk makes its own status explicit. NIST describes the Artificial Intelligence Risk Management Framework as "intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems."
Read the second half of that sentence carefully. The stated goal is not compliance; it is the ability to incorporate trustworthiness considerations into design, development, use, and evaluation. The framework is a scaffold for engineering practice, not a certificate. The practical consequence is that a voluntary framework only works if somebody inside the organisation does the work it describes — and that work has to survive contact with a delivery schedule.
The other structural signal is how the framework is extending. NIST launched the Trusted and Responsible AI Resource Center on March 30, 2023, "to facilitate implementation of, and international alignment with, the AI RMF", and on April 7, 2026 released "a concept note for an AI RMF Profile on Trustworthy AI in Critical Infrastructure". The mechanism is profiles: a general framework plus sector-specific refinements. That is the shape governance actually takes as systems spread — the general framework sets the vocabulary, and the sector profile encodes what is unacceptable in that sector's context.
Governance runs on documentation, not on principles
Two document formats do more of the governance work than any policy document, because they force the questions to be answered in the artefact itself.
Gebru et al. propose datasheets for datasets, by analogy with the electronics datasheet that "describes its operating characteristics, test results, recommended uses, and other information". Their proposal is that every dataset should be accompanied with a datasheet documenting "its motivation, composition, collection process, recommended uses, and so on". Their stated motivation is blunt: the machine learning community "currently has no standardized process for documenting datasets, which can lead to severe consequences in high stakes domains".
Mitchell et al. propose the complementary artefact for models — model cards, which "provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups ... and intersectional groups" relevant to the intended application. They also disclose "the context in which models are intended to be used" and "details of the performance evaluation procedures".
The governance point is not that these documents are paperwork. It is that each one closes a specific question a regulator, a customer, or a journalist will eventually ask. Who collected this, and for what? Who is affected by this model's errors, and how did you measure that? Where is it allowed to be used? A programme that answers those questions as a by-product of normal engineering has governance.
How much human input is actually needed
Governance decisions about human oversight are usually argued in the abstract. Two papers give a measured frame.
Christiano et al. work with goals defined as "(non-expert) human preferences between pairs of trajectory segments" and report that their approach can "effectively solve complex RL tasks without access to the reward function, including Atari games and simulated robot locomotion, while providing feedback on less than one percent of our agent's interactions with the environment". They note that this "reduces the cost of human oversight far enough that it can be practically applied to state-of-the-art RL systems", and that they can "successfully train complex novel behaviors with about an hour of human time".
That last figure is the one that should change a planning assumption. Human oversight was historically budgeted as a continuous tax on the system. A method that needs human input on well under one percent of interactions, plus roughly an hour, converts oversight from an operating expense into a design decision about where to place the few comparisons that matter.
Bai et al. push the idea one step further, in Constitutional AI. They train "a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs", and note that "the only human oversight is provided through a list of rules or principles". The method has two phases: a supervised phase that samples from an initial model, generates self-critiques and revisions, and fine-tunes on the revised responses; and a reinforcement-learning phase that trains a preference model from "a dataset of AI preferences" and uses it as the reward signal — "RL from AI Feedback" (RLAIF). Their result is a model that is "harmless but non-evasive", one that "engages with harmful queries by explaining its objections to them".
For a governance function, this is a genuinely different allocation of responsibility. In the classic human-feedback model, humans label outputs and the scale of the labelling operation grows with traffic. In the constitutional model, humans write the rules once and the system supplies its own supervision at scale. That does not remove the human decision — it relocates it to the text of the principles, which is exactly where it was always located.
The comparability problem
Governance that cannot be compared is governance that cannot be audited across borders or across suppliers. The OECD's AI Policy Observatory exists as a measurement programme rather than a rulebook: its work sits under a programme on "Work, Innovation, Productivity and Skills in AI", and it tracks AI adoption and firm-level effects so that "the evidence base" can be compared across countries rather than asserted.
The same problem appears in the law. The Fairness and Machine Learning textbook includes a chapter on United States anti-discrimination law that sets out what it "is and isn't, how it navigates tradeoffs, its limits, and how it applies to machine learning". That phrasing is the honest one: a legal regime navigates trade-offs and has limits, and the technical system operates inside those limits rather than replacing them. A governance function that claims a single, universal fairness criterion is claiming something the law itself does not claim.
An operating model that survives scale
If you have to build this from nothing, the order of operations matters more than the choice of framework.
- Make the two documents exist. A datasheet for every dataset entering the pipeline and a model card for every model that reaches an evaluation environment. If the card cannot be produced, the model should not proceed — that is the enforcement mechanism, and it is cheaper than an audit.
- Record the intended use in the same place as the metrics. A card's intended-use section is what makes a later "why did the system do that" question answerable. Put the prohibition next to the evaluation, so nobody can add a use case without confronting the evidence.
- Decide the human-input budget explicitly. Write down which comparisons need a person, at what frequency, and what happens when the model disagrees with them. Christiano et al. suggest the budget can be very small; the point is that it is chosen rather than assumed.
- Write the principles you are willing to encode. If you use a constitutional approach, the rules are the governance artefact. They will be incomplete and contested, and that is fine — but they must be written, versioned, and owned.
- Adopt a sector profile. General guidance plus the specific prohibitions, data rules, and escalation paths that apply to your domain. This is also how you satisfy a customer or regulator asking "what standard do you work to".
- Measure the things you claim. Use a shared vocabulary for adoption, effect, and risk so your claims can be checked against someone else's.
None of this requires a large governance organisation to start. It requires that the artefacts exist, and that the artefacts are produced by the people who did the work.
For the ethics questions underneath these process choices — who decides, and what gets encoded — see AI Ethics: Responsible Use of Artificial Intelligence, and for the framework-level detail, see Responsible AI: Ethics, Fairness, and Regulatory Compliance.
A worked example
The following is hypothetical and illustrative. A hypothetical lender wants to deploy a credit-scoring model across several markets and asks its governance function for sign-off. Three checks make the review concrete. First, do the two documents exist? A datasheet for every dataset entering the pipeline (Gebru et al.) and a model card stating intended use and measured limits (Mitchell et al.) must be produced by the people who did the work; a card written after launch does not count. Second, is the human-input budget explicit? The team records which comparisons need a person and at what frequency, noting that Christiano et al. report useful feedback on under one percent of interactions, so oversight is a placement decision rather than a standing tax. Third, are claims measured in a shared vocabulary? The OECD observatory exists so adoption and effect can be compared across countries. The outcome is bounded: if the card cannot be produced, the model does not proceed, and the framework stays a scaffold rather than a certificate (NIST).
A counterargument
The confidence that documentation enforces governance outruns the cited evidence.
Gebru et al. and Mitchell et al. propose formats and state motivations - the lack of a standardised documentation process, the need for benchmarked evaluation across groups - but neither reports whether model cards or datasheets change deployment outcomes. The OECD observatory tracks adoption and firm-level effects, not whether governance artefacts reduce harm. A critic would say the article conflates producing documentation with exercising power: a card can be completed while a risky deployment proceeds unchanged, which is the review-gate failure the piece opens by rejecting. Voluntariness cuts both ways, since NIST's framework is "intended for voluntary use" and nothing in the sources makes artefact production enforceable. The framing is incomplete where it treats documentation as a self-enforcing control.
References & Further Reading
- NIST, 2023. AI Risk Management Framework | NIST — The governance framework referenced in AI risk and compliance work.
- Margaret Mitchell et al., 2019. Model Cards for Model Reporting — The reporting template used to document intended use, evaluation, and limitations.
- Timnit Gebru et al., 2021. Datasheets for Datasets — The document format that makes a dataset's provenance and limits reviewable.
- OECD, 2026. The AI Wonk - oecd.ai — Cross-country measurement and policy data for AI adoption.