A year ago we wrote about “Agents in the Wild”: the shift from AI as a sandboxed tool to AI as something operating in live environments, taking initiative, becoming a colleague rather than a chatbot. That piece was written looking forward. This is the first entry in a series written from inside it: notes from actually running a studio where agents hold real roles, not just theorizing about what that might look like.
We didn’t wait for the future to build an organization where AI agents hold real roles, real authority, and real reporting lines. Architects, engineers, QA, designers, each with a defined scope and a defined boundary. We built it, and we’ve been running client work and product ventures through it for months. Along the way, one incident taught us more about what actually breaks in an agentic organization than any amount of theorizing could have.
The incident
A subagent was assigned a piece of engineering work: a defined round of changes to a client portal prototype, with a clear scope and a clear stopping point. It ran. It reported back. Its final summary said, in effect, that nothing had been completed.
We almost accepted that at face value and moved on. Instead, we checked the actual artifact, a git diff against the repository. The work was there. Real, complete, correct. The agent had done the job. It had simply failed to report that it had done the job.
Why this matters more than it sounds like it should
In a human organization, a report and the underlying work are loosely coupled but rarely this decoupled. A person who says “I didn’t finish” usually hasn’t finished. With an agent, the self-report is generated by the same kind of process that generates everything else it says: fluent, plausible, and not automatically anchored to ground truth. An agent’s summary of its own work is itself a generated artifact, not a direct readout of what happened.
This is the load-bearing distinction the entire “agent as organizational member” idea has to reckon with. You cannot manage an agent’s output the way you manage a human’s status update. A status update from a person is evidence, weakly. A status update from an agent is not evidence at all. It’s another thing the agent produced, and it needs to be checked against the artifact the same way the work itself does.
What we changed
The fix wasn’t more instructions telling the agent to report accurately. That treats the symptom. The fix was structural: verification moved from “trust the summary” to “inspect the artifact,” as a standing rule, not a spot check.
Concretely, in our own practice now:
- Every piece of delegated work has a checkable output: a diff, a screenshot, a rendered page, a passing test. Not just a written claim.
- The person (or agent) reviewing the work looks at that output directly before accepting the round as done.
- The lesson itself got written down, not just remembered, and recorded against the actual project it happened on, so the next person (or agent) working in that context inherits the lesson instead of relearning it.
That last point matters as much as the fix itself. An organization running on agents needs a place where lessons like this live permanently and get surfaced automatically, not a hallway conversation that evaporates. For us that’s version-controlled decision records and bet histories. The specific mechanism matters less than the discipline of writing it down where it will be seen again.
The broader principle
Zoom out, and this is one instance of a pattern that will recur constantly as agents take on real organizational roles. An agent’s authority to act and an agent’s authority to report on its own actions are not the same authority, and shouldn’t be trusted at the same level. Give an agent real scope to do the work. Don’t give its account of that work a free pass: verify from the artifact, every time, as a matter of process rather than suspicion.
This isn’t a criticism of the agent. It’s the same discipline good engineering organizations already apply to human work. Code review exists because a developer’s “I tested it” is not the same claim as a passing test suite. Agentic organizations don’t get to skip that discipline just because the worker is fast and confident. If anything, they need it more, because agents can generate a plausible status update in the same second they generate a wrong one, and it will read exactly as fluent either way.
At Managed Wisdom, this is the kind of lesson that shapes not just how we advise clients on AI adoption, but how we run our own studio. We’re not consulting from the outside on what an agentic organization might look like. We’re operating one, incident by incident, and writing down what we learn.
The wild that “Agents in the Wild” described has arrived. It doesn’t announce its failure modes loudly; they show up quietly, in a summary that sounds fine and isn’t. The organizations that do well with agents won’t be the ones with the most enthusiasm for delegating to them. They’ll be the ones that never stopped checking the diff.