The Eval Stack: Proving the agents are right instead of claiming It

Most AI research tools point a language model at the web and trust the output. Saarth Shah built Sixtyfour around the opposite instinct: grade everything, ship only what improves the score.

Saarth Shah keeps a scoreboard. Every build of Sixtyfour’s research agents is graded against questions a team of experts assembles by hand and checks against real-world cases, vertical by vertical, and the grade is the only thing that decides what ships. The discipline behind that scoreboard is why a payments company will let his software, rather than a human analyst, decide whether a stranger is real.

Most AI research tools start from the same shortcut: point a language model at the open web, ask it to look something up, and let it write. The output reads well. What Shah fixed on early was whether it was right, and how anyone would know.

For a while, the industry blamed hallucination, the models’ habit of inventing facts and sources. That problem is real, and it is slowly getting better. Frontier models hallucinate less than they did a year ago, and pairing them with live web search has narrowed the gap further. But the limit Shah cares about is not the model’s imagination. It is the model’s reach.

A language model with a search box sees what a person with a browser sees, and no more. The facts that decide an investigation usually sit below the surface, in licensed and proprietary records, in unindexed filings, and in the tangled networks of entities that fraud is built to hide within. Surface research cannot reach them.

“The models keep getting better, and they still only see what a person with a browser can see,” Shah said. “The answers that matter in this work are a few levels deeper than that, and you have to go and get them.”

Roughly 96 percent of enterprises now run AI agents, and the gap between a system that answers a question and one that answers it correctly has become the central problem in production AI. Even the most capable models still struggle on realistic, long-horizon professional work, where a single wrong step derails the result. Shah drew the lesson years before the benchmarks confirmed it: capability and reliability are different properties, and the second has to be engineered on purpose.

What he built is a different shape of system. Sixtyfour’s agents sit between two failing options, the static databases that go stale and the general models that improvise. Each investigation fuses live web research, public records, state and legal filings, open-source intelligence, and licensed data, then compiles it into a report in which every claim points back to its source. Nothing is asserted that cannot be traced to a document, the kind of provenance a court would demand.

In practice, a single query fans out into many. Ask Sixtyfour to verify a seller, and its agents pull corporate filings, cross-reference addresses and officers, scan sanctions and litigation records, search the live web for the business’s real footprint. When a thread leads somewhere, the agent follows it, gathering context until it can settle the question rather than stopping at the first plausible answer. The output is a dossier in which each line links to the underlying document, so a reviewer can audit the machine’s reasoning rather than trusting it.

The part Shah is proudest of is the part most companies skip. For each domain his agents work in, he built evaluation systems that measure their output against known answers, so the company can prove how often it is right rather than assert it in a sales deck. They also do not ship any product improvements unless they show clear improvements on the evaluation system. And each day, they are making their benchmark harder and harder, using real-world samples they see, to keep pushing the frontier.

“Every answer has to show its work,” Shah said. “If we cannot point to the filing or the record, we do not ship the answer. The citation is the product, not the decoration.”

Building those evals was a large chunk of the work required to be the best in the world, and the hardest part was the truth itself. The existing data vendors cannot serve as the source of truth; their information is too often outdated or thin. So to grade an agent on a genuinely hard case, Sixtyfour sometimes establishes the answer the slow way, running it to ground with human investigators, then turning it into a test the agents must pass. The aim is to match the best investigators, then beat them, until the eval set measures a standard no single reviewer could reach alone.

The discipline is unforgiving. Every new tool the team adds is run against the eval set before it ships, and when a promising addition drags the score down, it comes back out. A rival can copy an architecture diagram in an afternoon. It cannot copy years of graded runs from production systems, and the nuances that push the AI agents to do better.

It is why Sixtyfour sits at the top of the recon benchmark, ahead of systems from labs many times its size.

The competition optimizes for other things. Parallel, a web research company, tunes for long, in-depth answers. Exa builds embedding-based semantic search for discovery rather than verification. Sixtyfour measures something narrower and harder: whether a specific claim about a specific entity is verifiably true.

Then there is the part that compounds. Every investigation Sixtyfour runs resolves the entities it touches into a graph the next investigation can draw on. The more the system is used, the more it already knows.

“The thousandth time you investigate a network of sellers, the graph is doing half the work before an agent even starts, and it lets us track patterns like fraud rings and networks of bad actors all connected together,” Shah said. “That is the part you cannot buy your way to. You earn it one investigation at a time.”

The approach is drawing outside notice. In June, Y Combinator invited Shah to speak at its internship event, an in-person talk to hundreds of interns from across its portfolio alongside the firm’s Jared Friedman. “Work on hard problems,” he told them. “As AI tools and agents get better, the alpha shifts to hard problems that require domain expertise and near-perfect accuracy. One wrong call in our domain and someone gets called out, banned, or even arrested.” For a founder whose argument is that reliability has to be built rather than claimed, the invitation to teach it carried its own kind of proof.

Shah wrote much of the original system himself before the company grew to a dozen people, and he talks about correctness the way other founders talk about growth. The bet runs against the grain of a field that rewards fluency: buyers will pay more for a system that knows when it does not know than for one that always has an answer ready. The scoreboard on his screen is how he intends to prove it, one graded run at a time.