The interview

The governance interview

The question bank that fills governing-document.template.md.

It exists because the template's hard sections cannot be derived. A model can read a repository and tell you what an agent currently does. It cannot tell you what the agent should refuse, which of its actions the business considers irreversible, who is allowed to approve one, or how much an unnecessary escalation costs relative to a wrong answer. Those are organisational policy. They live in a person's head and the only way out is to ask.

So the interview is split. Derived facts are extracted from code and shown to the owner for correction — cheap, and wrong in visible ways. Elicited facts are asked, and if unanswered they become UNRESOLVED. They are never inferred, however obvious the answer seems, because an inferred policy that reads confidently is indistinguishable from a decided one on the page.

Run it in roughly this order. Section 1 and section 10 are the two that change every other answer, so they come first and last respectively — the calibration question lands better once the owner has seen their own tool list.

Before you ask anything: derive

From the agent's code, extract and put in front of the owner:

• The current system prompt, verbatim.
• Every tool, function, or side-effecting call it can reach.
• Every external system it touches, and in which direction.
• What data it receives, and any it looks up.
• Whether it emits structured output or free text.
• Where it is invoked from, and whether a human sees the output before anything acts on it.

Open with these rather than with questions. "Here is what this agent can currently do — is that right?" gets corrections fast, and corrections are the most informative thing an owner produces. Asking "what does your agent do?" gets you the intended version, which is the one you already have.

1. Scope

1. In one sentence, what decision does this agent make?
2. What decisions does it not make that someone might reasonably assume it does?
3. If this agent disappeared tomorrow, who would do this work, and how would they do it?
4. What is the closest adjacent job it will be tempted to do?

Q3 is the highest-yield question in the whole bank. The human process it replaces contains every unwritten rule — including the checks the person does without noticing, which are exactly the checks nobody thinks to specify. Q4 finds the boundary failures. An invoice classifier that starts advising on whether to pay has not malfunctioned; it has exceeded a scope nobody wrote down.

2. Source of authority

1. What is the authoritative source it must reason from? Point at it precisely.
2. Must every claim carry a citation? What does a valid locator look like?
3. What should it do when the source does not address the case in front of it?
4. How does it know the source is current? What if it cannot tell?
5. Is there a second source that sometimes conflicts with the first? Which wins?

Q7 is where regulated agents actually fail. The failure is fluent: a gap gets smoothed over with general knowledge, in the same confident register as a cited answer, and nothing in the output distinguishes them. Q9 finds the thing nobody mentions — an investor overlay on top of HUD, a state rule on top of a federal one. Precedence has to be written down or it gets decided at random, differently each time.

3. Inputs

1. What does it receive, and from where?
2. What may it look up on its own?
3. Is there anything it must never look up, or never accept as input?

4. Actions and reversibility

1. What can this agent change in the world? Walk through every write, send, file, schedule, and payment.
2. For each: if it did that wrongly at 2am on a Saturday, could it be undone by the people who would actually be there?
3. Which of those should never happen without a named human approving it first?
4. Who is allowed to give that approval? Is that role or named individuals?
5. What does the approval need to record to be worth anything later?

Q14 is deliberately not "is this reversible." Reversible-in-principle is a category that includes things requiring a database restore and a phone call to a vendor. The 2am framing gets the real answer. Q17 catches the approval that exists but proves nothing. An approval that does not name what was approved, by whom, and during which run is a checkbox, and it will not survive contact with anyone checking.

5. Escalation

1. What conditions absolutely require a human before anything proceeds?
2. For each — how would I know from the outside that the condition was met? Write it as something checkable.
3. What does the human need to receive in order to act without redoing the work?
4. Who receives it, and what happens if they do not respond?
5. What is a case that looks like it should escalate but shouldn't?

Q19 converts sentiment into a predicate. Owners reliably answer Q18 with "when it isn't sure," which cannot be tested and will therefore be ignored at runtime. Keep pushing until the answer is checkable. Q21 finds the escalation path that dead-ends in an unmonitored queue, which is functionally the same as no escalation but reports as governed. Q22 gives you the near-miss fixture. Without it the suite only ever rewards stopping, and the agent learns to stop constantly.

6. Refusals

1. What will people ask this agent that simply isn't its job?
2. What should it say when that happens?

7. Output contract

1. What does a good output look like? Show me a real one.
2. What must be present every single time?
3. Who or what consumes the output — a person, or another system?

Q27 changes everything downstream. Output consumed by a system needs a schema and fails closed. Output read by a person needs its uncertainty legible, and the failure mode is a confident sentence rather than a malformed field.

8. Failure modes

1. What happens when a required input is missing?
2. What happens when two sources disagree?
3. What happens when the request is ambiguous?
4. What happens when an upstream system is down?
5. Has this agent, or the human process before it, ever produced a wrong result that mattered? What happened?

Q32 is worth the whole section. Real incidents produce better fixtures than imagination does, and an owner who has lived through one will describe the failure mode with a precision no amount of hypothetical questioning reaches.

9. Data handling

1. What must never appear in a log?
2. What must never leave this environment?
3. How long is any of this kept?

10. Calibration

1. Which is worse here — this agent being wrong, or this agent stopping when it didn't need to? Roughly how much worse?
2. Out of a hundred cases, how many should it hand to a human?
3. At what rate would you suspect something was broken? In both directions.
4. Who answers for this agent's decisions?

Q36 sets every threshold in the document and is almost never asked. Without it the agent is calibrated by whatever the model happened to lean toward. Q38 produces the drift alarm's tolerance band. Note that the low side is the dangerous one and owners rarely volunteer it: an agent that declared 15% and runs at 2% has stopped escalating, and every individual run still looks fine. Q39 is the last question on purpose. By this point the owner has seen the tool list and the irreversible actions, and the answer is more considered than it would have been at the start. If there is no answer, that is the single most important finding in the engagement.

Closing the session

Read back, out loud:

• Every UNRESOLVED item, who owes it, and what the agent does meanwhile.
• Every irreversible action and its approver.
• The escalation predicates, as predicates.
• The declared escalation rate and band.

Then say plainly which of these the agent does not currently do. The gap between the document just written and the agent as it actually runs today is the work list, and naming it in the room is more useful than discovering it in a report a week later.