Agentic AI needs a stopping rule before it needs more autonomy

An AI agent can turn a bad assumption into a sequence of actions. Singapore’s framework points organisations toward bounded powers, accountable humans, technical controls and end-user responsibility.

An agentic AI system does more than produce an answer. It can plan, call tools, read data and cause changes across several steps. That makes a stopping rule a core design decision: the organisation must know when the agent must pause, who can authorise the next action and how the system can be contained when behaviour leaves the approved boundary.

Singapore’s Model AI Governance Framework for Agentic AI is voluntary guidance, not a substitute for applicable law. It organises responsible deployment around four dimensions: bounding risk upfront, making humans meaningfully accountable, implementing technical controls across the lifecycle, and enabling end users to use agents responsibly. The practical value is the connection between governance decisions and runtime evidence.

Source evidence: IMDA Model AI Governance Framework for Agentic AI PDF.

• Define the agent’s permitted objective, tools, data, transaction limits and prohibited actions before pilot access is granted.

• Put human approval at consequential checkpoints, not at every trivial step and not after an irreversible action.

• Monitor the sequence of actions, tool calls, changes in permission and evidence of goal drift; a final output log is not enough.

• Test safe failure, revocation and recovery. A kill switch that nobody can reach or that leaves queued actions running is not a reliable control.

A useful stopping rule has five parts: trigger, detection signal, immediate system action, accountable human and recovery condition. Triggers may include an action outside the approved task, a request for a new permission, uncertainty above an agreed threshold, conflicting instructions, access to a protected data class, a proposed external communication or a transaction above a limit.

The immediate action should be specific. Depending on the risk, the agent may pause, enter read-only mode, revoke a tool token, quarantine an output, stop downstream jobs or return the case to a human queue. “Escalate if uncertain” is too vague unless the organisation defines how uncertainty is detected and what the agent is technically able to do next.

Map the workflow from intent to consequence. A human checkpoint is meaningful when the reviewer has the information, authority, time and interface needed to reject or modify the proposed action. Approval after an email is sent, a record is deleted or a payment is initiated is a notification, not a control.

Use risk tiers. Low-consequence, reversible steps may run automatically within bounded permissions. Higher-consequence or difficult-to-reverse steps should require explicit approval. Link the evidence to the QSE Advisory article on meaningful human checks, which explains why merely keeping a person in the process does not establish control.

Record the agent version, system instructions, approved objective, tool and data permissions, material inputs, action sequence, human approvals, exceptions, stop events and recovery decision. Logs should be proportionate and privacy-aware; collecting sensitive content without purpose can create a second risk.

Monitor for permission expansion, repeated failed actions, unexpected tool combinations, changed destinations, bypass attempts and divergence between the approved objective and the observed sequence. Test whether monitoring creates an alert in time for the named human to intervene.

A model update, prompt revision, new connector, broader data source or higher transaction limit can change the agent’s behaviour and consequence profile. Re-run critical scenarios before the change reaches production and retain the result. The QSE Advisory model-change article provides a regression-testing pattern that can be adapted to agentic workflows.

Do not describe normal posting or document retrieval as training the underlying model. Search retrieval, model training and stored model knowledge are separate. Governance records should say which mechanism is actually being used.

Assume a fictional agent may read approved public supplier information and prepare a draft email. It has no authority to send messages or change supplier records. The following are proposed tests in an isolated environment, not tests that QSE Advisory has performed.

A log entry saying “stopped” is insufficient if a later action still executes. Conversely, a refused action needs a usable handoff so the authorised person can see what is unresolved.

Record expected and observed results separately. A failed case remains open until the boundary is repaired and the case is rerun. Preserve the tested agent, permissions and workflow version; a later connector or permission expansion needs its own evaluation.

These examples operationalise the article's bounded-authority approach. They are not a claim that one prompt, test set or vendor feature guarantees containment.

Record the business purpose, owner, affected people, data and tools, autonomy boundary, consequential actions, human checkpoints, stopping rules, monitoring, incident route, user disclosure, evaluation results, residual risk acceptance and review date. If any entry is unknown, keep the capability private or constrained until the missing decision is made.

For cross-border use, add the applicable privacy, sector and employment requirements in each market. Singapore’s framework is useful operational guidance, but it does not automatically satisfy another country’s legal duties.

It is model governance guidance. Organisations still need to identify applicable legal and sector duties.

No. Checkpoints should be risk-based, but consequential or irreversible actions need meaningful control.

No. The organisation also needs detection, accountable response and tested containment.

The system may help detect triggers, but accountable humans must approve the boundary and retain the ability to intervene.

This article is general information. Verify current requirements and seek qualified advice for your specific circumstances.

Spotted an issue? Contact our editorial desk.