A check that can't fail isn't a check: what coding agents miss

Coding agents now write a large share of the API code that ships. Ask why an agent chose an endpoint, a field or an auth flow, though, and there is usually no answer. The agent did not record its reasons, and nobody checked what it was reading.

At apisec|con 2026, Sterling Chin, Founder of Sannr and former head of Postman Labs, where he led the R&D team behind Postbot and Postman Flows, took that question head on. His answer was blunt: "...do we really know why it chose what it did? And the answer is no." (06:08)

His talk split the problem in two. Before the agent works: what context did we give it, and is that context true? After the agent works: what did it learn, and where did that knowledge go? Below are the ideas from his session that matter most for security leaders, with our take on each.

A green check that can't fail proves nothing

In early tests of his tool, Sterling pointed an agent at 10 APIs. Every call came back 401, unauthorized. Every one was recorded as verified, with a green check none of them had earned.

"A check has to be able to fail. A check that cannot fail is not really a check, and that's something that an agent will still trust, and it'll trust it exactly as if it was a real check." (27:29)

This is the line every security leader should take from the talk. As agents write code and then verify their own work, a dashboard full of passes can describe a system nobody actually tested. In our view, a result only counts when the test could have failed, ran against the live application, and left evidence someone can replay. Run it twice, get the same answer.

Coding agents never ask for the context they need

Sterling's next finding came from hundreds of experiments. He gave coding agents every way to look things up: databases, MCP servers, vector stores, even a dedicated "librarian" agent that answered well when asked. They never asked.

"Coding agents are so confidently oblivious that they never reach for context. They never reach for anything." (12:27)

He compares agents to junior engineers who fill gaps with assumptions instead of asking the question that matters. His fix was to put context right inside the code, next to what the agent is changing. In his own cold-versus-warm test (18 sessions each, his figures), agents that started with notes from earlier sessions ended with 6 persisted incidents; agents that started cold ended with 81.

For security teams, the lesson is that an agent works with whatever is in front of it. Context is the home-field advantage, but only when it is actually delivered.

Did the call authenticate, and did the scope match?

Sterling's playbook has three steps: deliver, confirm and test. Confirm, he says, is where the security lives.

"Did the call actually authenticate? Did the write actually refuse where it should have? Did the scope the agent used match the scope that you gave it?" (27:56)

Those are facts an agent can record and a reviewer can check before anyone trusts a result. His minimum bar is simple: put a date and a source on every API fact, even in a wiki, and mark whether it was observed or only written down.

We would add one point. Agents act with real credentials against running applications, so authentication, authorization and scope are exactly where proof matters most. Proving what an agent could actually reach is the same discipline as proving what an attacker could.

Treat agent context like a lock file

The third way context goes bad, after staleness and plain error, is planting. A single line slipped into a document or data set can steer every agent that reads it. Sterling expects this to become an attack vector, if it hasn't already.

"Context files are now our supply chain, okay? So we need to treat them like a lock file." (29:28)

His controls are ones security teams already run for code: review every change like a pull request, put code owners on the folder, and pin each note to a digest so a quiet change fails the check.

Agents, MCP servers and the files they read now ship inside the applications we are responsible for. You cannot defend what you cannot see, so the first step is knowing where that context lives.

"Check the docs if you need it" never works

Because agents never go looking, Sterling's first rule is to build for the push, not the pull.

"So my point here is: if you're building agent tooling, build for the push, not the pull." (26:00)

Every harness reads one file before it does anything else. Put the pointer there, not in a wiki. If the context lives in the repository, it ships on every branch and every clone, and every agent gets it for free.

Every closed session loses what it learned

Sterling's sharpest leadership point was about memory. When an engineer gave notice, managers used to panic and ask them to write everything down. Today, every closed agent session throws away what that agent learned, and nobody writes anything down.

"So why aren't we panicking about the knowledge we lose at the end of every session?" (10:29)

Most organizations already struggle to say what applications they have deployed and what those applications are made of. Code that agents write and nobody can explain widens that gap.

What this means for your program

1. Find out where your agents and their context live. Inventory the coding agents, MCP servers and instruction files in your repositories. apisec's free, open-source AI Surface and MCP audit tools map the AI and MCP systems in your code.
2. Make every check able to fail. Ask your teams to show a test that went red before you trust one that is green. A pass with no possible failure is a gap, not a control.
3. Record the security facts of every agent call. Did it authenticate? Was a production write refused where it should be? Did the agent stay inside the scope you granted?
4. Put context under change control. Review instruction and context files like code, with diffs, named owners and pinned versions.
5. Push context, don't wait for agents to pull it. Keep it in the repository and point to it from the one file every agent reads first.

Proof, not green checks

Sterling's talk is about coding agents, but the principle reaches every part of application security. AI writes more of the code, AI finds more of the flaws, and a result is only worth trusting if it was proven against the running application. Attackers can afford to be probabilistic. Defenders can't.

That is the idea behind Application Exploit Validation: see every way in, prove what an attacker can actually use, and confirm the fix.

Watch Sterling's full session above, then try apisec free to see what an attacker could reach in one of your own applications.

Frequently asked questions

Why don't coding agents use the documentation or MCP tools we give them?

In Sterling Chin's experiments, coding agents never reached for available context, even when a dedicated agent was ready to answer. Like a junior engineer, they fill gaps with assumptions. His advice is to put context directly in front of the agent, inside the code it is working on, and load it into every session.

What is provenance for API context?

Provenance means every API fact carries who, what, where, when and why: who verified an endpoint, what was sent and returned, where it was last used, when it was last verified and why the endpoint exists. Sterling's minimum bar is a date and a source on every fact, marked as observed or only written down.

How do you know an agent's "verified" result is real?

A check has to be able to fail. In Sterling's early tests, calls that all returned 401 were still recorded as verified. In apisec's view, a result is only trustworthy when it ran against the live application, could have failed, and left evidence that can be replayed and gives the same answer twice.

Can the context agents read be attacked?

Yes. Sterling expects planted context, a quietly changed line that steers the next agent, to become an attack vector. He recommends treating context files like a lock file: review changes as diffs, assign code owners, and pin each note to a digest so an unreviewed change fails the check.