Model Provenance and Citation, Explained

What provenance actually means

The word predates AI by a long way, and the pre-existing definition is the useful one. The W3C PROV Data Model, a W3C Recommendation published 30 April 2013, defines provenance as "a record that describes the people, institutions, entities, and activities involved in producing, influencing, or delivering a piece of data or a thing." It organizes that record around three concepts: entities (the things produced or used), activities (the processes that used and produced them), and agents (whoever or whatever is responsible for an activity).

Map that onto a model output and the picture gets concrete quickly. The entity is the paragraph the system wrote. The activity is the generation run — a specific model, a specific prompt, a specific set of retrieved passages, at a specific time. The agents are the model and the system that invoked it. The entities used are the source records. Provenance, in this sense, is not a feeling of trustworthiness. It is a set of identifiers you can resolve.

That is the standard worth holding, because it separates two questions people tend to merge. "Is this output correct?" is a question about the world. "Can I tell where this output came from?" is a question about the system. The second is answerable by engineering; the first usually is not. A system with real provenance does not promise you the right answer — it promises you can find out.

A citation is a claim, and claims can be wrong

Here is the mechanic that surprises people the first time. A language model is trained on enormous quantities of text, and a great deal of that text contains citations — case reporters, journal references, footnotes, bibliographies. The model learns the form of a citation extremely well: the shape of a case name, the plausible volume-and-page range, the way a DOI is punctuated. Producing that form is a generation task. Verifying that the referenced document exists, and that it says what the sentence claims, is a retrieval task. Nothing about learning the first forces the second.

NIST gave this failure mode a name. In its Generative AI Profile (NIST AI 600-1), published July 2024 as the cross-sectoral companion to the AI Risk Management Framework, NIST lists confabulation — confidently stated but erroneous or false content — as one of twelve risk categories unique to or exacerbated by generative AI. A fabricated citation is confabulation in its most legible form, because unlike a wrong date or a garbled number it comes with its own built-in verification handle.

The best-documented public example is a federal court case. In Mata v. Avianca, Inc., Judge P. Kevin Castel of the Southern District of New York sanctioned two attorneys and their firm on 22 June 2023 after they filed a brief built on six judicial decisions that did not exist. The detail worth noticing is not that the cases were invented — it is how thoroughly they were invented. The Clerk of the Eleventh Circuit confirmed that the lead fabricated decision was not an authentic ruling of that court, and that no party by that name had appeared there since the court adopted electronic filing in 2010; the docket number the fake decision carried belonged to a real but entirely unrelated case. The fake decision even contained internal citations to four further cases that also did not exist. The output was structurally perfect and referentially empty.

Three different things get called "a citation"

When a vendor says a system "cites its sources," they may mean any of three quite different things. Knowing which one you are being offered is most of the analysis.

1. Generated reference text. The model writes a citation the same way it writes any other sentence. Nothing checked that the referent exists. This is the Mata failure mode. It is cheap, it looks identical to the real thing, and it is worth nothing.

2. Retrieved-and-attached sources. The system searches a corpus, puts the retrieved passages in the model's context, generates an answer, and attaches the retrieved documents as citations. Now the documents exist — a real improvement. But the attachment is between the answer and the retrieval set, not between a specific sentence and the specific passage that supports it. The model may have used a retrieved passage, ignored it, or blended it with its own prior knowledge.

The gap is measurable. Nelson Liu, Tianyi Zhang, and Percy Liang audited four generative search engines by human evaluation in "Evaluating Verifiability in Generative Search Engines" (Findings of EMNLP 2023; arXiv:2304.09848). They report that on average only 51.5% of generated sentences were fully supported by their citations, and only 74.5% of citations supported the sentence they were attached to. Every one of those systems was retrieving real documents.

Retrieval alone does not fix it in specialist domains either. In "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools" (Journal of Empirical Legal Studies, 2025; arXiv:2405.20362), Varun Magesh, Faiz Surani, Matthew Dahl and colleagues at Stanford and Yale ran a preregistered evaluation of retrieval-based legal research products from LexisNexis and Thomson Reuters. They found those tools hallucinated between 17% and 33% of the time — against vendor claims of "hallucination-free" citations.

3. Span-level attribution. Each statement carries a machine-checkable pointer into a specific region of a specific document the system was given: a character range, a page range, an index into a content block. You can resolve the pointer mechanically, read the span, and decide for yourself whether it supports the sentence. This is the only one of the three where the citation is a handle rather than a claim.

How to verify one

Three checks, in order, and they take about a minute each.

Does the referent resolve? Take the identifier and look it up in the system of record — not in the AI system, and not in a web search that may surface the AI system's own output. If the identifier does not resolve, stop; nothing downstream matters.

Does the cited span actually state the claim? Open the passage and read it against the sentence. The academic literature has a name for the property you are testing: AIS, or "Attributable to Identified Sources," introduced by Hannah Rashkin and colleagues in "Measuring Attribution in Natural Language Generation Models" (Computational Linguistics 49:4, 2023). Their test is a good one to borrow: a statement counts as attributable when a generic hearer, given the source, would affirm "According to [the source], [the statement]." A passage that is merely on the same topic fails. So does one that supports a weaker version of the claim.

Could the system have produced this citation without reading the source? This is the structural question, and it is the one that generalizes. If the reference was generated as text, the answer is yes, and you are back at level one no matter how the interface renders it. If the reference is an offset emitted alongside a document the system was handed, the answer is no — the pointer cannot exist unless the document did.

Do this on a real sample, not on a demo. Ten sentences drawn from your own corpus, checked by hand, will tell you more about a system than any benchmark run on someone else's data. And measure per sentence, not per answer: the Liu et al. numbers above look very different at the two granularities, and the sentence is where a reader's trust actually lives.

What the plumbing looks like when it is real

Span-level citation is not exotic; it is a documented API feature at several vendors, and reading one implementation makes the concept concrete. Anthropic's Citations documentation describes the shape well. You mark a document block with citations: {"enabled": true}; the service chunks the document into sentences; the response comes back split into text blocks, and cited blocks carry a citation array. Each citation carries the quoted source text plus a location object whose type depends on the document:

• Plain text — a char_location with 0-indexed start and end character offsets (end exclusive).
• PDF — a page_location with 1-indexed page numbers, after text extraction and sentence chunking.
• Custom content — a content_block_location, a 0-indexed range over the blocks you supplied, with no further chunking. This is the option when you want to control granularity yourself, as with transcripts or bulleted records.

The mechanical property that matters: a character offset into a document you supplied is not a sentence the model composed. It is an index. You can slice the source with it in one line of code and compare. That is what makes span-level attribution auditable at scale rather than one paragraph at a time — which is exactly what an accreditation reviewer, an inspector general, or a peer reviewer is going to want when they ask how a conclusion was reached.

Where span-level citation stops working

Every honest description of this machinery includes the places it does not reach, and the vendor documentation above is refreshingly direct about two of them: citing images inside PDFs is not supported, and a PDF that is a scan with no extractable text is not citable at all. If your corpus is scanned paper, the citation layer cannot be built until the OCR layer is. That is an ingest problem wearing a provenance costume, and it is very common in records that predate digital filing.

Three more limits are worth stating plainly. Offsets prove a passage was available and pointed at; they do not prove the model's reasoning over that passage was sound — synthesis across five cited records can still be wrong while every individual citation checks out. Chunking granularity sets a floor on precision: if a document is chunked into paragraphs, you get paragraph-level pointers, and a reader still has to find the sentence. And provenance says nothing about whether the source itself is correct — a perfectly traced citation to a superseded regulation is perfectly traceable and perfectly wrong.

A note on scope, since this is what we build. Precision Federal (Precision Delivery Federal LLC, an Iowa limited liability company) builds small models that read through a body of data and produce a written conclusion with every statement traced back to the exact record it came from. Three things we will not do, listed here because they are the questions worth putting to any vendor, including us. We will not present a retrieved-and-ranked passage as a verified citation — those are level two and level three, and the difference is the whole point. We will not quote an accuracy or attribution rate for your corpus that we have not measured on your corpus. And we will not build a span-level citation layer on top of a corpus that cannot support one; if the records are scans without text, or transcripts without stable offsets, the honest answer is to fix the ingest first and say so, rather than ship a pointer that looks like a proof.

Sources: W3C — PROV-DM: The PROV Data Model (Recommendation, 30 April 2013); NIST AI 600-1 — Generative Artificial Intelligence Profile (July 2024); Mata v. Avianca, Inc., No. 1:22-cv-01461 (S.D.N.Y. June 22, 2023) — opinion and order on sanctions; Liu, Zhang & Liang — Evaluating Verifiability in Generative Search Engines (Findings of EMNLP 2023; arXiv:2304.09848); Magesh et al. — Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (Journal of Empirical Legal Studies, 2025; arXiv:2405.20362); Rashkin et al. — Measuring Attribution in Natural Language Generation Models (Computational Linguistics 49:4, 2023); Anthropic — Citations documentation. Analysis and framing by Precision AI Academy.

Common questions

What is provenance for an AI output? The record of where the output came from: which source records were used, what process consumed them, and who or what performed that process. The W3C PROV Data Model (a W3C Recommendation from 30 April 2013) defines provenance as a record describing the people, institutions, entities, and activities involved in producing, influencing, or delivering a piece of data. Applied to a model output, that means every sentence should be traceable to the specific records that produced it.

What is the difference between a real citation and a plausible-looking one? A real citation points to a source that exists and contains the claim being made. A plausible-looking one is a well-formed string that satisfies a reader's expectation of what a citation looks like, without either property having been checked. A model can produce the second without the first, because generating correctly formatted reference text and retrieving a real supporting passage are different operations.

Does retrieval-augmented generation eliminate fabricated citations? No. Retrieval makes the cited document real, but it does not guarantee the document supports the sentence attached to it. In a preregistered evaluation published in the Journal of Empirical Legal Studies, Stanford and Yale researchers found retrieval-based AI legal research tools from LexisNexis and Thomson Reuters produced incorrect information between 17% and 33% of the time, despite marketing describing them as avoiding or eliminating hallucinations.

How do you verify a citation from an AI system? Check three things in order. Does the cited record exist as an identifier you can resolve in the system of record? Open the cited span and read it — does it state the claim, or merely sit near it? And could the system have produced that citation without reading the source, which is the case whenever the reference is generated as text rather than emitted as an offset into a document the system was given?