Working with a system you cannot fully audit

You have delegated work to software for a long time. What is different is that you are now delegating judgement — not “run these steps” but “work out what the steps should be.” A compiler you do not read is deterministic and will make the same choice twice. An agent is neither.

Three things follow, and they compound:

• Volume. It produces more than you can review line by line, so “I read everything” stops being an available strategy.
• Accumulation. It builds up context — conventions, history, decisions — that you never explicitly gave it and do not hold yourself.
• Opacity of reasoning. You can read what it did. Why it chose that over the alternative is reconstructed after the fact, by the same kind of system, and is not a recording.

None of this is unprecedented — it is roughly the position of anyone who manages people whose specialism they do not share. It is simply arriving faster, at lower cost, and with far less social friction to slow it down.

This is the load-bearing principle and everything else follows from it. When your agent deploys the change, the deployment is yours. When it emails the customer, your company sent that email. When it deletes the database, you deleted the database.

“The agent did it” is not available as an explanation to a customer, a regulator, or a colleague — and you would not accept it from someone else. The capability is delegated; the responsibility is not.

If you cannot review every action, review the things that still fit in your head. Attention is the scarce resource; spend it where it changes outcomes.

Boundaries are the highest-leverage of these because they are small, stable, and reviewable in minutes. A credential scope is a sentence. The work it enables is thousands of actions you will never read.

You will not achieve a system that is always right. You can achieve one where being wrong is survivable. Prefer the arrangement you can undo over the one you are more confident in — confidence is exactly the thing that fails quietly.

In practice this is unglamorous: pull requests rather than direct pushes, staging before production, infrastructure defined in code so it can be rebuilt, backups you have actually restored from, and a rollback path identified before the change ships rather than during the incident.

Worth noticing early, while adjusting is still cheap:

• You approve its work without reading it, because it has been right before.
• You cannot say what it currently has access to without going and looking.
• Something in production exists that nobody on your team can explain the reason for.
• You have stopped asking it to explain, because the explanations were long and it was usually fine.
• You would not know for days if it started doing something wrong.

None of these is a failure of the agent. They are all drift in the relationship, and the remedy is the same: narrow the scope, restore a checkpoint, and re-establish what you actually know versus what you have been assuming.

Trust an agent the way you would trust a colleague: specifically, not globally. Someone excellent at refactoring has not thereby demonstrated judgement about customer communications. Competence in one domain is weak evidence about another — and an agent's fluency makes that easy to forget, because it sounds equally assured everywhere.

Widen scope in response to demonstrated results in that kind of work, and be willing to narrow it again. Scope is a dial, not a ratchet.

Responsible operation is not only your side of the arrangement. Our obligations, so you can hold us to them:

• Isolation you can rely on. Your agent runs on its own instance, under its own role, with its own encrypted secrets. It is not a tenant in a shared process.
• Your credentials stay yours. They are injected into your agent's tasks and are never logged, never returned by an API, and never reachable by another customer's agent.
• Revocation that actually revokes. Deleting an agent destroys its instance, its secret, and its storage. Deactivating a key in your own IAM cuts access immediately, with no cooperation required from us.
• Honest failure. When something is unavailable we would rather show you an error than a confident zero. A blank number that means “not configured” is worse than useless.

Keep the boundaries small enough to hold in your head. Prefer reversible over certain. Verify outcomes against reality rather than against reports. Extend trust for specific work, not in general. And keep owning the results — because you do, whatever the tooling.