How do you stop an AI research agent from hallucinating?

Make fabrication structurally impossible rather than discouraged. In our system every recorded finding requires a source and URL field, checkable claims live in a separate ledger with verification state, the agent is forbidden from producing hypotheses at all, and an empty run is a valid outcome. An agent under pressure to produce something will invent something — removing that pressure is the fix.

What is the difference between a finding and a claim?

A finding is what a source actually said, stored with its source, URL, confidence, relevance and timestamp. A claim is a checkable assertion extracted from findings, stored with verification_needed and verification_outcome fields so it moves through a verification state machine before anyone relies on it. Our 721 findings produced 310 claims — fewer than half survived as checkable.

Why separate the research agent from the analysis agent?

Because an agent allowed to both observe and interpret produces output where you cannot tell which is which. Ours is explicitly barred from theorising — "you produce hypotheses NEVER" — and a separate agent reads across the accumulated corpus over time to find patterns. Its output feeds a human approval gate, not the publishing pipeline.

Do confidence scores on AI agent output actually mean anything?

Only if calibration is enforced. Our research agent is instructed to tag every finding and claim and explicitly not to tag everything high, because a confidence field where every value is "high" carries no information. We enforce it as a rule rather than trusting the model to self-moderate — though we have not published a formal calibration study.

Research Agent With a Claims Ledger

The insight: separate what you saw from what it means

Most research agents produce one blob of output that mixes three different kinds of statement: things a source actually said, things the agent inferred, and things the agent invented to make the paragraph flow. Once they are in the same paragraph you cannot tell them apart, and neither can anyone downstream.

So we split them at the storage layer, before they can contaminate each other. Two append-only ledgers with different schemas and different rules.

The two ledgers

Findings — what a source said. 721 entries at survey time. Every one carries:

topic · body · source · url · confidence · relevance · ts

Claims — checkable assertions extracted from findings, tracked through verification. 310 entries. Every one carries:

claim · topic · source_url · confidence
verification_needed · verification_outcome
extracted_at · verified_at

The two fields that make this more than bookkeeping areverification_needed and verification_outcome. A claim is not published because it was extracted — it is published because it was extracted, flagged for checking, checked, and the outcome recorded. There is a state machine here, not a text file.

Note the ratio: 721 findings produced 310 claims. Fewer than half of what the agent read survived as a checkable assertion. That gap is the discipline working.

The four hard limits

The research agent's own instructions read like a compliance document, deliberately. Verbatim:

"Cite everything, and never fabricate." A finding with no source cannot exist — the schema will not hold it.
"You do not generate findings from no data." If the collectors fetched nothing, the correct output is nothing. An empty run is a valid run. This is the rule that most prevents invention, because the pressure to produce is what causes hallucination in the first place.
"You produce hypotheses NEVER — that's dreamer's." Interpretation is a different agent's job. The researcher is not allowed to theorise, so its output stays separable from analysis.
"You tag every finding and every claim. You do not tag everything high."Confidence scoring is worthless if everything is confident. Calibration is enforced as a rule, not left to the model's mood.

None of that is a prompt trick. It is a mandate plus a schema that refuses malformed entries. The agent cannot fabricate because there is nowhere to put a fabrication.

Where the data comes from

Two collection runs go out every night — an RSS pass at 01:30 and a GitHub pass at 01:45 — each following a documented runbook rather than a freeform instruction. A separate collection agent covers community sources with its own runbooks per platform: Hacker News, Reddit, Lobsters, dev.to, and GitHub trending, plus procedures for sentiment analysis, trend detection, and pain point extraction.

Everything they gather lands in the same two ledgers under the same rules. There is no side channel where uncited material can enter the system.

What happens after: interpretation as a separate role

Because the researcher is forbidden from theorising, we needed something that does. A separate agent reads across the accumulated ledgers over time and looks for what the daily view cannot see: "You connect findings from this week with patterns from last month. You notice what's quiet that used to be loud, what's loud that used to be quiet, and what keeps coming back."

It carries five runbooks for different kinds of pass — a retrospective, and four styles of traversal across the corpus including a drift walk and a tangent walk. Its output feeds the approval gate, not the publishing pipeline, so an interesting pattern still has to be approved before anything is built on it.

That is the full separation: one agent that may only report what it saw, another that may only interpret, and a human gate between interpretation and action.

What this case study does not prove

It does not prove zero hallucination. It proves fabrication has no valid storage location and that claims carry verification state. A model can still summarise a real source badly.
It does not prove the confidence scores are well calibrated. We enforce that not everything is tagged high; we have not published a calibration study.
It does not prove research quality. 721 findings is a volume measure, not a value measure.
It does not include client research. This is our own corpus. Client work is under NDA.

What we would carry into your build

If an agent's output is going to inform a decision, the architecture matters more than the prompt. Separate observation from interpretation into different agents. Give every recorded fact a mandatory source field so an uncited claim is structurally impossible. Track verification as state rather than as prose. And make an empty run a legitimate outcome, because an agent that must produce something will invent something.

That is a data-layer answer to hallucination, and it works regardless of which model you run.

A research agent that cannot fabricate