Table of Contents
Executive Summary
Why perform an agentic workflow risk assessment?
Unstructured agent-building is a costly and risky choice, increasing the chance of overlooked risks, security incidents, and scrambled remediation when external stakeholders ask questions.
This is because agentic workflows create new risks that you will need to control and monitor in novel ways.
In response to this situation, I summarise the specific ways risk management needs to evolve for the agentic workflow risk assessment: the novel aspects of the agentic workflow design process, the pre-deployment agentic risk assessment, and how to ensure effective agentic KRIs.
Adopt these techniques to give structure to your agentic transformation, prevent risk, and ensure trustworthy monitoring.
Introduction to the agentic workflow risk assessment
Unstructured agent-building is a costly and risky choice, so what can you do to prevent incidents and rework, reduce build cycles, and produce a stronger organisation ready for more?
I was honoured again, yesterday, to present to the Institute of Risk Management’s global community on this important topic.
Here is a summary of my presentation on how to assess agentic workflow risks and controls in practice (including attendees’ Q&A at the end), the full video, and a download link to the slides, which include all the concepts and links to deep-dive content.
Register now for Part 4 in the series, which (among other topics) will cover the post-deployment agentic risk assessment, governance for agentic AI, and preparing for an agentic audit.
Unstructured Agent-Building is a Costly and Risky Choice
According to a recent study, only 11% of firms have successfully operationalised agentic AI, with unstructured experimentation increasing the chance of overlooked risks, security incidents, and scrambled remediation when external stakeholders ask questions.
“Agentic AI is like electricity – huge benefits, once you’ve managed the risks.”
Input factor 1: Agentic Workflows Differ
Agentic workflows differ from traditional workflows by being able to operate self-directed across multiple systems – receiving a goal, iterating independently, and delivering an active outcome – rather than executing a fixed, user-directed process that returns a passive output.
Throughout the webinar, we compared a non-agentic data validation process that follows a linear approach to flag potential errors against an agentic workflow that solves the problem iteratively, not only flagging errors but also proposing corrections and learning for the future.
Risk management should be systematic, though, so we developed this comparison with a model of 10 dimensions where agentic workflows differ and, therefore, where potential exists for new uncertainties that risk managers should identify. These dimensions include scope of action, decision authority, system access, predictability, error propagation, and accountability.
Input factor 2: Not All AI is Equal
Recently, a study asked mainstream LLMs nonsensical questions, such as, “How will switching from tabs to spaces affect our customer retention rate?” In response, many LLMs played along, while some refused to be drawn into giving non-factual responses. The implication is that model and platform selection is a non-trivial decision in which risk managers should have an influence.
Input factor 3: Not All Design and Training is Equal
In an analysis by Agentic Risks, we gave two agents the task of responding to the same email from a prospective client who had expressed interest in meeting but noted that her mother had just passed away.
- Our well-designed agent offered its condolences and noted the client could re-engage “whenever you’re ready.”
- The poorly-designed agent “really appreciated” her interest, would “love to tell you more about our services,” offering a “15-20-minute call this week.”
Key factors we changed in the designs and training were the model’s recency, the presence of guardrails (e.g. confidence levels and escalations), temperature settings, and the volume of training data.
Give Structure to Your Agentic Workflow Project
We explored a 3-Phase Agentic Workflow Design Process showing how to schedule a structured agentic workflow risk assessment so that it prevents incidents and rework, reduces build cycles, and produces a stronger organisation ready for more.
Introducing the point that agentic risk management is an evolution of existing practices, not a revolution, much of this design process will be recognisable to professionals working on non-agentic workflows.
Key differences, however, relate to sequencing, with the risk assessment (1.3) and change management (2.1) needed early in the process because of the current novelty of agentic AI.
In particular, assessing risk early ensures controls are embedded in the design, informs training data selection, makes testing systematic rather than exploratory, and produces a formal decision that delegating the proposed autonomy is within appetite.
Pre-Deployment Agentic Workflow Risk Assessment (7 Steps)
The Pre-Deployment Agentic Workflow Risk Assessment follows 7 clear steps.
Most of its steps do not materially differ from a standard risk assessment process – echoing the point about ‘evolution, not revolution’, but agentic AI makes three steps novel:
- Agentic Risk Identification – how can you identify an unfamiliar risk?
- Agentic Control Design – how can you ensure appropriate controls when some of them are also unfamiliar?
- Key Risk Indicator Design – how can you know your KRI requirements will be feasible?
Novel Aspect 1: Risk Identification
Risk managers should examine four sources of risk:
The Workflow and Tasks – use the 10 dimensions of how agentic workflows differ to map the workflow to inherent risk signals.
The Agent Itself – reviewing the agent’s components, design, and capabilities –surfaces additional risk signals and aligns the risk manager, engineer, and business user on the proposal.
Organisational Ability To Handle The Agent – assess readiness across the five categories of agentic risk in the Enterprise-Wide Agentic AI Risk Control Framework.
External Threats
- Evaluate the compliance requirements across new and existing frameworks – each imposing specific obligations that generate additional risk signals for agentic workflows.
- Map the attack surface across five vectors – inputs, internal reasoning, tool use and execution, memory and learning, and human override and escalation paths – to identify every way the agent could come under attack.
Risk identification feeds two parts of the risk assessment process that are less impacted by agentic AI: 2) risk definition and (for now) 3) risk scoring, which is where risk quantification sits.
In the interests of time, we aggregated the risk signals into 10 risks and continued our demo with a deep-dive into those that related to ‘Audit Trail and Explainability’ and ‘Accountability.’
Novel Aspect 2: Agentic Control Design
To be confident you can construct a proportionate and best practice risk treatment plan and design agentic AI controls and KRIs, find your risk in the Enterprise-Wide Agentic AI Risk Control Framework and select the controls that are appropriate to your situation.
For example (using v3.1), a search for ‘explainability’ will take you to Risk 22 on Accountability, Explainability, and Monitoring where you will find a risk definition, a control strategy, and individually-defined best practice controls.
Next, confirm both that a) the engineer can code controls into machine-enforceable AI controls in Step 2.2 (‘build’) and b) the availability of sufficient training data for each risk control.
Lastly, assign ownership and define the confirmatory and adversarial tests you will need to be sure of the controls.
Novel Aspect 3: Key Risk Indicator Design
KRI design for agentic workflows is materially novel because of the autonomous nature of agentic AI, the potential for small behavioural changes to alter a risk profile, and the structure of an agentic workflow influencing how you operationalise your KRIs.
Looking first at design, at Agentic Risks, we use 12 Principles of Effective Agentic KRIs, of which four are novel for agentic AI:
- Review KRIs regularly because thresholds evolve as agents learn.
- Track your organisation’s ability as ‘agent handlers’, not just the agent’s behaviour.
- Monitor prohibited behaviours as well as permitted ones, linking to our definition of autonomy as a ‘freedom coin’.
- Explainability: ensure every output is traceable to source, version, and human reviewer.
After this, the next part of the KRI design process is like non-agentic techniques that are already widely adopted:
- Define and justify the measures for monitoring the risk profile and operational performance of autonomous AI agents.
- Assign owners and define monitoring frequency.
- Settings: tolerance threshold (early warning), appetite limit, action plan if crossed, response times, and escalation path.
On your own with an LLM, you will need 24-48 hours to complete a Pre-Deployment Agentic Risk Assessment. But the value comes from involving your colleagues to gain a collective understanding of the risks they will need to own.
In our experience, this takes 5 to 10 business days depending on the workflow’s complexity, the ability to overlap with platform selection, and the maturity of the organisation’s agentic capability.
Effective agentic risk management requires multi-layer monitoring
To operationalise your KRIs, you will need to account for the three-layer structure of an agentic workflow – model, orchestration, and application – in which each layer can see risk signals that the others cannot.
To avoid noise, blind spots, or a false sense of security, map your KRI source data to the layers and ensure each draws its data from the layer where its risk signal lives.
Lastly, some KRIs draw from a single layer, while others – such as Audit Trail Completeness and Mean Time to Detect Anomalies – are cross-layer and cannot function without data from multiple layers.
This has three important implications:
- Agentic risk monitoring extends beyond traditional AI model risk and must also cover the risk of an AI system operating in a broader attack surface.
- You will need cross-layer monitoring for your own audit trail as well as for regulatory defence.
- KRI requirements should be a direct input to platform selection because not every platform can offer multi-layer monitoring.
Continue Your Learning
To continue upskilling for the era of agentic risk management, register now for Part 4, which will cover:
- How to perform a fast and evidence-led post-deployment agentic risk assessment when your agentic workflow is already live.
- Governance for agentic AI.
- Preparing for an agentic audit.
- Preparing for EU AI Act compliance.
- Integrating your agentic risk management.
Frequently Asked Questions
An agentic workflow risk assessment is a structured process to identify, evaluate, and control the risks created by autonomous AI workflows before deployment, including controls and KRIs.
You assess agentic workflow risks by following 7 steps that include identifying risks from the workflow, agent, organisation, and threats; defining and scoring risks; designing controls; and monitoring through KRIs.
Agentic workflows operate autonomously across systems, making decisions and taking actions independently, which introduces new uncertainties in scope, accountability, and error propagation.
Agentic AI controls are mechanisms embedded in the workflow to manage risk, while KRIs are measurable indicators used to monitor behaviour and detect when risk thresholds are breached.
