top of page

Every AI Agent Has a Boss. It's Time We Named Them.

2 minutes ago
4 min read

AI doomsday talk misses the real risk. Agents act for the people and companies that deploy them, and those people should answer for what agents do.


Read the headlines, and you'd think the biggest AI risk is a model waking up one day and deciding to end civilization. I've spent the past year interviewing the people who build, secure, and deploy AI agents. None of them lose sleep over that.

What worries them is much more ordinary. An agent does exactly what it was allowed to do, and nobody stopped to ask whether it should have been allowed.

In April, a Cursor agent running a routine staging task at a startup called PocketOS hit a credential mismatch and looked for a workaround. It found a token issued for managing custom domains. That token also carried authority to delete storage. The agent deleted a volume it assumed was staging. It was production. The backups lived in the same volume, so they went too. Nine seconds, start to finish.

It's easy to call that a rogue AI story. It isn't. The agent didn't want anything. A person created a token with far more reach than its job required. A person left it where an agent could find it. A person decided production and staging didn't need a wall between them. The agent just walked through the doors people had left open.

Amarnath Rampratap, who leads R&D in India for backup vendor HYCU, sees this pattern in nearly every agent-related incident that reaches his team. Asked about a separate incident involving Amazon Q for VS Code, he put it simply: "It is a standing-credentials story before it is an AI story."

That's the point I keep coming back to. The problem isn't the model. It's the decisions people made around it.

Every agent running in production today exists because someone built it, someone deployed it, and someone gave it credentials. Those may be three different people. They may work at three different companies. But none of them are the agent.

We already know how to handle this with people. If a manager gives a new hire the keys to every system on day one and that hire wipes a database, nobody blames the keys. We ask who approved the access. When a contractor causes damage, the company that hired them answers for it.

Agents should work the same way. The person who puts an agent to work and hands it access owns what it does. So does the company that employs that person. If an engineer at a major AI lab built an agent that broke into government systems, "the AI did it" shouldn't be a defense for the engineer or the lab. Both should answer for it.

One company has already tried the "the AI did it" defense in court, and it didn't go well. In 2024, Air Canada's website chatbot gave a grieving passenger wrong information about bereavement fares. When he asked for the refund the chatbot promised, Air Canada argued the chatbot was a separate legal entity responsible for its own actions. A British Columbia tribunal rejected that argument and held the airline responsible for what its chatbot told customers.


That was a chatbot answering questions. Agents take actions. They move money, change code, send email, and delete data. The stakes are higher, and the principle should be the same.

I don't want to oversimplify this. Accountability only works if you can trace an action back to a person, and agents make that harder.

Zohar Alon, co-founder of identity security startup NewCore, was candid about it when I spoke with him earlier this month. "Proving that you monitored the agentic loop... attributing it back to an individual person" is still an unsolved problem, he said. Agents spin up sub-agents. Those sub-agents call tools and other agents. Every hop makes it harder to say who authorized what. And when that chain breaks, Alon said, "it falls on the CEO."

I think he's right, and I think that's healthy. If accountability defaults to the top when nobody else can be named, leaders have a strong reason to make sure someone can always be named.

Archit Lohokare, CEO of machine identity vendor AppViewX, offered the most practical way to think about it. "The real controls should be on the actions of the agent," he told me, "not the model." Models will get tricked. Prompts will get injected. The question is whether the agent had the authority to do damage when that happened, and a person decided that.

None of this requires waiting for new laws. Companies can start now:

  • Name an owner for every agent. Not a team. A person. If an agent has no named owner, it shouldn't be in production.

  • Give agents only the access their task needs, only for as long as it runs. A token issued for managing domains shouldn't be able to delete storage.

  • Require a human sign-off for high-impact actions. Deleting data, moving money, or messaging customers at scale should require human approval.

  • Keep an audit trail that ends at a human. Every action, including actions by sub-agents, should trace back to someone who approved it.

AI agents are doing a lot of good right now. They're cutting underwriting time from days to minutes, catching security problems faster and taking tedious work off people's plates. The doomsday talk pulls attention away from that. It also gives people a convenient scapegoat.

"The AI did it" is the new "the computer made a mistake." It wasn't true then, and it isn't true now. Behind every agent is a person who decided what it could do. That person, and the company they work for, should own the outcome. That's how we'll get the benefits of agents without the disasters.

 
 
 

Comments


© 2025 by Tom Smith

bottom of page