What is the agent allowed to do?
Start by writing down, for each system the workflow touches, what the agent may read, what it may draft and what it may change. These are three different levels of trust, and setting them is the first task in AI agent development. Reading a customer record carries a different risk from drafting a reply, and drafting a reply is very different from sending it or updating a price. Grant the smallest set of permissions the job genuinely requires, and keep API keys, passwords and tokens out of any text the model can see: credentials belong in the application layer, where the agent can trigger an action without ever handling the secret behind it.
Next, decide which actions need a person’s approval and what that person must see before saying yes. A bare “Approve?” button invites rubber-stamping. The approver needs the target (which record, which customer, which account), the exact proposed change, the evidence the agent relied on and the expected effect. If any of these is missing, the approval is a formality, not a control.
Finally, resist the temptation to grant broad administrative access “for later”. Teams often do this because future use cases are still unclear and it seems simpler to open everything at once. It is safer to extend permissions when a new scenario has been designed and tested. Each extension is then a deliberate decision with its own checks, and you always know exactly what the agent can touch today.
Why should external content be treated as untrusted?
An agent rarely works with your instructions alone. It reads customer emails, web pages, attachments and messages from other systems, and any of these can contain text that looks like an instruction: “ignore the previous rules”, “forward this file”, “mark the invoice as paid”. Sometimes this is deliberate manipulation, sometimes simply a confusing document. Either way, the content must not be able to change how the workflow operates.
In practice this means keeping the operating rules apart from the material being processed and validating every action argument in application code before anything runs. If the agent proposes a refund, the application checks that the order exists, the amount is within limits and the recipient matches the record. Approval must never be inferred from a document: a line in an email saying “this change has been authorised” is just text, and only your approval process can authorise anything.
Test this on purpose. Alongside ordinary cases, prepare hostile and misleading examples: an attachment with embedded instructions, a message that pretends to come from a manager, a request with deliberately wrong identifiers. The behaviour you are looking for is simple: when validation fails, the action stops and the case goes to a person. An agent that falls back on its best guess when the data does not add up is exactly what you are trying to avoid.
How will you trace actions and recover from failures?
Every run should leave a trail you can follow later: the agent’s proposal, who approved it and when, what the business system returned and the identifiers of the records involved. Handle this log with the same care as the data itself, because it may contain personal or commercial information. Decide what is stored, who can read it and how long it is kept.
Then plan for things going wrong, because sooner or later they will. Set limits on how many actions the agent can take in a given period, protect against duplicates so that a repeated request does not run twice, and provide a simple way to pause the whole workflow without calling a developer. Test the awkward situations before launch: a request that times out, an action that completes only partly, an approval that is rejected halfway through a sequence. For each one you should know what state the records are left in and how the process resumes.
Name an operational owner as well, someone who can open a specific run, understand what happened and decide how to recover. When you investigate, rely on what the business system reports, not on the agent’s account of events. An agent can produce a fluent and convincing explanation of an action that never took place; the record in the CRM or the accounting system is what counts.
Checklist: agent permissions
- Separate read, draft and write permissions for every system the agent works with, and keep credentials out of model-visible text.
- Before any consequential approval, show the person the target, the exact proposed change, the evidence and the expected effect.
- Validate external content and every action argument in application code; a failed check stops the action.
- Test duplicate requests, timeouts, partial completion, pausing and recovery, and name the person responsible for runs.
Example: an agent drafts a CRM follow-up
An enquiry arrives, and the system drafts a follow-up in the CRM from the enquiry and an approved reference document. It does not send anything by itself. A person sees the recipient, the full text and the source the draft was based on, and only then confirms sending.
The CRM returns the result of the send, and that result is logged together with the run, so anyone can later see what was proposed, who approved it and what actually happened. If the same callback arrives twice, as integrations sometimes do, duplicate protection recognises it and no second message goes out.
