Skip to main content

What "verbatim" means when the source is a PDF

· 4 min read
The MindMax team
The MindMax team
Building MindMax

Checking that an AI's quote really appears in a PDF sounds like pressing Ctrl+F. In practice the text inside a PDF is not the text you see: words are hyphenated across lines, paragraphs are split by page breaks and running headers, and letter pairs like "fi" are stored as single symbols. A quote check that ignores this either rejects real quotes or accepts paraphrases. Getting it right is most of the work.

We said every quote in MindMax is checked against the source. This post is about why that sentence hides more than it shows — and what you should expect from any tool that claims the same.

A PDF is a drawing of text​

A PDF describes where to paint characters on a page. It does not, in general, record sentences or paragraphs. Reading text out of it is reconstruction, and the reconstruction has predictable seams:

  • Hyphenation. "infor-" at the end of one line and "mation" at the start of the next. The sentence the model read says "information"; the raw text says "infor- mation".
  • Ligatures. Typeset "fi", "fl" and "ffi" are often single glyphs. A word that looks like "efficient" can come out of the file as something that does not match "efficient" letter for letter.
  • Page breaks. A paragraph that runs from page 12 to page 13 is two fragments in the file, with a page number, a running header and maybe a footnote in between.
  • Columns and figures. Two-column layouts with a figure caption in the middle are where reading order goes wrong most often.
  • Quotation marks and dashes. Curly versus straight quotes, hyphens versus en dashes. Models tidy these constantly.

Two ways to get it wrong​

A quote checker has two failure modes, and they pull in opposite directions.

Too strict, and real quotes fail. The model quoted the sentence correctly; the check rejected it because of a hyphen at a line end. Users learn that "not found" means nothing, and stop looking at the flag.

Too loose, and paraphrases pass. "The treatment may reduce risk" matches "the treatment reduces risk" closely enough to slip through. That is the failure the check existed to catch.

The job is to forgive the page — hyphens, ligatures, headers, quote styles — while staying strict about the words. A dropped "may" has to fail.

What we do about it​

Without going into the machinery, here is the behaviour you can see:

  • Paragraphs are rebuilt across page breaks, so a quote is not cut in half by the bottom of a page or polluted by the running header above it.
  • A quote is matched against the document's words, not its typesetting, so line-end hyphens and ligatures do not cause false alarms.
  • A near miss is put back into the source's own wording. If the model tidied punctuation or swapped a word, the citation you see uses the document's exact text — and that exact text is what you are judging.
  • A quote that cannot be found is never shown as checked. In an answer it is flagged at the end; an AI margin note whose quote does not match is sent back and redone.
  • The citation lands on the sentence, highlighted on the page, so the last check — does this sentence support the claim? — takes one look.

What to expect from any tool​

If you are evaluating an AI reader, try this: take an answer, pick a quote that sits at a line break or a page break in the PDF, and click it. Does it land? Then ask about a passage you know is hedged, and see whether the hedge survives in the quote. Those two checks tell you more than any accuracy claim.

What is not claimed​

Extraction is not perfect. Dense mathematics, two-column layouts with interleaved figures and scanned pages with poor recognition are still where quotes are least reliable, and where clicking through to the page is worth the second. We are not claiming every quote is right; we are claiming a wrong one is shown as wrong.

The idea in full is checked quotes; why it matters is in why AI invents citations. The reader itself is on the AI PDF reader page.