How AI Learns Changes What You Need to Prove

A business wants to use AI to manage customer service requests. One system predicts which team should receive a request. Another groups customers with similar behaviour. A third learns a policy for choosing routing actions.

Each supplier may present a strong test result. For an AI governance professional, the next question is what that result establishes. The answer depends partly on how the system learned. Labels, discovered patterns and rewards each carry an assumption that a performance score may leave untested.

The learning paradigm does not determine whether a system should be approved or how much authority it should receive. It helps identify the evidence needed to make that decision.

Supervised learning: examine the answer the model was taught

A supervised model learns from examples paired with answers, usually called labels. In this hypothetical service system, past requests might be labelled with the team that handled them. A test can then measure how often the model predicts that team for requests it has not seen before.

But the historical team may not have been the right one. If urgent complaints were routinely sent to a general queue, a model could reproduce that practice accurately. Its test score would describe agreement with the records, not whether customers received appropriate help.

The assurance work begins with the target: who assigned the labels, what did they represent, and do they correspond to the outcome the organisation wants? It also requires looking at the errors that matter. Misrouting a routine enquiry and delaying an urgent complaint should not disappear into a single accuracy figure.

For governance, the challenge is to establish whether predicting the recorded answer supports the proposed decision.

Unsupervised learning: examine the meaning assigned to a pattern

An unsupervised model can find structure in data without being given a correct answer for each example. It might group customers by similarities in their requests and service histories.

There are ways to assess properties of those groups. A clustering measure might show that members are close together according to the features and method chosen. Such a measure has assumptions and limits, as the scikit-learn documentation[1] explains. It cannot tell the business what a group deserves.

That step belongs to people. If a manager names one group “low value” and proposes reducing its service, the organisation needs evidence for both the interpretation and the action. A domain specialist should examine actual cases, the features behind the grouping, its stability and the consequences of putting someone in the wrong group.

Exploring a pattern may be reasonable with limited evidence. Using that pattern to change a customer’s treatment requires more. The governance challenge is to establish when an observed similarity becomes a justified basis for action.

Reinforcement learning: examine what the reward encourages

Reinforcement learning trains a policy to choose actions using a reward signal. Suppose a routing policy is rewarded for reducing handling time. Transferring difficult requests quickly could improve that measure while leaving customers without an answer.

A test of the reward alone would miss the problem. The organisation would need to examine resolution, repeat contacts and actions the system must never take merely to improve its score. Research on concrete problems in AI safety[2] identifies poorly specified objectives and unwanted side effects among the concerns associated with optimisation.

The reviewer also needs to establish when the policy can change. It may be trained before deployment and updated only through controlled releases. It may instead learn during operation. Those arrangements call for different monitoring and change controls; the term reinforcement learning does not tell us which one applies.

Here the governance challenge is to establish whether improving the reward improves the intended outcome within acceptable limits.

From learning method to an assurance decision

A real system may involve more than one learning method. A large language model can have several training stages, then be connected to retrieval, tools and business permissions. Its training history helps identify what to ask, but the organisation must also test the system as it will actually be used.

The NIST AI Risk Management Framework[3] connects evaluation to intended use, operating conditions, monitoring and responsibilities for intervention. For a governance professional, that turns technical evidence into a decision: what has been demonstrated, which cases remain uncertain, what may the system do, and who can act if the evidence stops holding?

Learning paradigms are useful because they expose different ways a test result can be persuasive without answering the question that matters. Assurance starts by finding that unanswered question.

References

[1] scikit-learn developers. Clustering: clustering performance evaluation (documentation, version 1.9.1 inspected). 2026. https://scikit-learn.org/stable/modules/clustering.html#clustering-performance-evaluation

[2] Dario Amodei and co-authors. Concrete Problems in AI Safety. 2016. https://arxiv.org/abs/1606.06565

[3] NIST. AI Risk Management Framework Core. 2023. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/