Table of Contents
One year ago, ‘agentic risk management’ wasn’t a thing, so when a few of us concluded this would change, we were able to get our hands on a stack of related web addresses because no one else seemed to want them.
Roll forward 12 months, and we now run a popular webinar series for the Institute of Risk Management; we are The Investment Association’s approved training provider on agentic AI; we blog regularly about the ‘human-agent organisation’; we have been asked to speak at industry events; and we have completed some major client engagements on agentic risks and controls.
It goes without saying, therefore, that these experiences have taught me a huge amount. So, on our first birthday, here are the top 10 agentic AI lessons I have learned over this period.
If you are delegating autonomy to agents at your firm (or soon will be), then this is for you.
For centuries, the organisation has been a human-only structure. Agents bring the first non-human delegated workforce, introducing a new paradigm: the human-agent organisation. Enter it with confidence.
1. Compliance is not control
The lesson. The EU AI Act, ISO 42001 and NIST AI RMF were not designed specifically for autonomous agents that produce outcomes in our name. A firm can therefore pass its AI compliance audit and still hold an agent estate it cannot constrain, track, or stop.
Why it matters. This decides where you start. Treat agentic AI adoption as a compliance exercise and you will assemble the evidence a regulator asks for, while leaving the operational risk untouched. Agentic AI governance has to reach beyond the standards, because the standards do not yet describe what an agent does.
2. Your roadmap must be achievable from your current state of readiness
The lesson. The firms whose adoption strategies succeed are those whose roadmaps start with a clear view of where they actually are, not want to be. This is because binding organisational constraints – accountability, data flows, workflow design, workforce capability – hinder agentic deployment before the need for new technical skills.
Why it matters. An agent does not compensate for a weak process; it runs that process faster and in more places. Sequencing a roadmap from assessed readiness costs less than discovering the gap mid-deployment.
Popular service > Agentic AI Readiness Assessment
3. Autonomy is a scale you calibrate to your needs, and where you set it is a risk appetite decision
The lesson. Autonomy runs on a scale, from tightly constrained systems to highly autonomous ones. Each use case should carry only the autonomy that its stakes, uncertainty, reversibility and safeguards justify.
Why it matters. Left implicit, the setting is made by whoever configures the agent. Taken explicitly, within board-approved agentic AI risk appetite and applied per use case, it gives project teams a boundary to design against and gives you a defensible answer to how you decided. This is the decision an executive team has to make together, and early.
Popular service > Agentic Transformation Initiation Workshop
Snack on the top 10 agentic AI Lessons through this carousel:
4. Tier your agents on objective criteria, including how well you can control them
The lesson. A low-risk task performed by an agent you cannot constrain, track, or stop is not a low-risk agent. Tiering should test data sensitivity, autonomy level, control strength, system access, multi-agent orchestration and external exposure, then apply proportionate treatment to each tier.
Why it matters. Tiering is what lets you move quickly and safely at the same time: register-and-attest for a large low-risk fleet, lightweight assessment in the middle, and full pre- and post-deployment risk management for the few agents that warrant it.
5. Design the controls before you choose the platform
The lesson. Run your agentic AI risk assessment once the workflow design is known and before platform selection. That sequence tells you which controls the platform must support, which the engineer must embed, what the training data must contain, and what testing must prove.
Why it matters. The alternative is accepting whichever controls the chosen platform happens to offer. Assessing after selection reduces testing to exploratory probing rather than systematic evaluation of controls and KRI thresholds, and leaves rework as the only remedy.
6. Put the safeguards outside the model
The lesson. A control the model can reason its way around is not a control. Serious deployments need four independent layers: hardcoded constraints, output verification, reversibility by design, and a tested escalation path.
Why it matters. This is the shift from descriptive governance – “we have a policy that prohibits X” – to operational governance – “the system cannot do X, because of control Y”. The second one produces evidence. Naming the four layers for a specific workflow is the work that turns a policy into a control set.
Popular service > Agentic AI Control Design Workshop
7. Monitor continuously where the risk signal lives
The lesson. An agentic workflow has three layers – model, orchestration and application – and each sees signals the others cannot. A KRI is viable only when it draws data from the layer where its signal resides, and some KRIs need more than one.
Why it matters. This makes KRI design a prerequisite of platform selection rather than a consequence of it. Misplaced indicators produce noise, blind spots, or a false sense of security, and monitoring designed after deployment inherits whatever the chosen tools happen to expose.
8. Data must be fit for autonomous consumption
The lesson. ‘Clean data’ is the standard we inherited from reporting. An autonomous consumer needs data that is SCARVeS: data structured for retrieval, current under version control, authoritative with expert provenance, rich in reasoning and escalation conditions, verifiable against ground truth, and symmetrical, covering failure modes as well as ideal cases.
Why it matters. Agents propagate what they read. An agent relying on superseded policy acts consistently with the wrong policy, and training data showing only success teaches an agent nothing about when a human would have paused.
9. Accountability should be architected: a named owner, a governed identity, defined approval points
The lesson. Every agent needs a named human manager, its own unique identity with least-privilege and revocable access, an auditable trail, and explicit points where a human must block, approve, or override. Escalation is an operating capability you will need to staff and train.
Why it matters. An escalation path without enough context, authority or capacity transfers risk rather than managing it. Escalate too often and you have recreated the manual process; too rarely and the agent is operating beyond its competence
10. Govern human and agent operations as a combined and ongoing capability
The lesson. Governance for agentic AI is not a project with an end state, and running parallel regimes – one for people, one for AI – leaves the handoffs between them ungoverned. Combined operations in a human-agent organisation need a single framework, with humans retaining ultimate decision rights, and a funded plan for ongoing review, retraining and decommissioning.
Why it matters. Agent behaviour can change after deployment, so governance that closes at launch can quickly end up describing a system that no longer exists. Treating it as a permanent capability is also what makes an agentic advantage durable.
Frequently Asked Questions
Agentic AI risk management is the discipline of identifying, assessing, controlling and continuously monitoring the risks created when AI agents can act on an organisation’s behalf. It focuses not only on what a model outputs, but on what an agent can access, decide and do, how it is constrained, and when humans must intervene.
Traditional frameworks remain useful, but they were not designed for systems that can pursue goals autonomously and create outcomes in our name. Agentic AI therefore needs operational controls that can constrain, trace, monitor and stop agents, not only policies and periodic review.
Treat autonomy as a scale. Give each use case only the autonomy justified by its stakes, uncertainty, reversibility, safeguards and board-approved risk appetite; as risk or loss of controllability rises, tighten limits and human approval points.
Once the workflow design is known and before the platform is selected. This lets control requirements influence the platform, engineering, training data and testing, rather than forcing the organisation to accept the controls a chosen platform happens to provide.
Continuously, across the model, orchestration and application layers. KRIs should draw from the layer where the relevant risk signal lives, with escalation and corrective action when behaviour changes or thresholds are breached.
AI agent accountability requires a named human manager, a unique governed identity, least-privilege and revocable access, an auditable trail, and defined human approval or override points. The agent may act, but humans retain ultimate decision rights.
A human-agent organisation is an organisation in which people and AI agents operate as one governed system rather than under parallel regimes. Humans retain ultimate decision rights, while agents receive delegated authority within defined controls, monitoring and escalation.
