# Screenshot a chart and ask AI? Why a region beats a crop

URL: https://mindmax.me/blog/screenshot-chart-ask-ai/
Last verified: 2026-10-08

> Pasting a chart into a chat window throws away three things you still had. What a boxed region carries instead, and why numbers come from the text, not the picture.

## It looks like a screenshot. It is not one.

September 22, 2026 · 6 min read

[The MindMax team](https://mindmax.me/about/)

Building MindMax

Asking AI about a screenshot of a chart works, but it throws away the text under the picture, the page
it came from and the document around it. Boxing a region inside the document keeps all three, so
numbers are read from the text rather than guessed from pixels, and the answer links back to the page.

Everybody has the same workaround. You hit a figure you cannot parse, you screenshot it, you paste
it into a chat window, you type "what is this saying". It works often enough to become a habit.

It also fails in a specific, repeatable way, and the failure is not the model being bad at
pictures. It is that by the time the picture arrives, three things that were sitting right next to
it have been thrown away.

## What the crop drops[​](https://mindmax.me/#what-the-crop-drops)

Open the figure in the actual document and look at what is there. The axis labels are text. The
caption is text. The table under the chart is text, with the exact values in it. The page number
is known. The paragraph that explains the figure is two inches above it.

Crop to a PNG and all of that is gone. What reaches the model is a grid of pixels in which the
number `0.37` and the number `0.87` differ by a few dozen of them.

So the model does what it must: it reads the picture. Most of the time it gets the number right.
Sometimes it gets it nearly right, which is worse, because a number that is nearly right does not
announce itself. And the answer that comes back has nowhere to point — you cannot click it, the
conversation has no idea which document that was, and the follow-up question ("what does the
paragraph above it say?") is unanswerable without another screenshot.

## What we send instead[​](https://mindmax.me/#what-we-send-instead)

In MindMax you drag the box inside the reader, and four things travel with it.

**The image**, because some things genuinely are pictorial: which box is under which heading, what
the arrows connect, how the layout groups things. This is what the picture is _for_.

**The verbatim text underneath the box**, harvested out of the document rather than recognised out
of the pixels. This is the part that does not exist when you paste a crop into a chat window.

**Where it came from** — file, page, chapter, slide. The answer can therefore be clicked back to
the spot, and the thumbnail in the conversation reopens the document at the right page.

**The document itself**, attached to the question, so the model can go and read the surrounding
section rather than speculating about the caption.

## The rule that does the work[​](https://mindmax.me/#the-rule-that-does-the-work)

Having both the image and the text is not automatically an improvement. The model still has to be
told which one wins, because it will happily read a value off a chart while a table containing
that exact value sits in its context.

So the instruction is explicit: where a region has a real text layer, **numbers, formulas and table
values follow the text, and the image is only for layout, shapes and spatial relationships.**

The interesting part is the other branch. Scanned pages, diagrams with the labels drawn in, and
figures in image-only PDFs have no text underneath. We could have papered over that — transcribe
the region silently and present the result as if it were source text. We do the opposite: when
there is nothing under the box, the model is required to say that a figure was read off the image
and may be wrong.

That branch matters more than it looks, because it is the one case where the honest answer is
_less_ impressive than the dishonest one. A tool that always sounds equally confident has told you
nothing about when to check.

There is a third case, and it is the one that caught us out during implementation. If you ask a
follow-up question two turns later, the image is no longer in the conversation history — history is
text. Our first fix was to keep a transcription of the region around instead. Which means that on
turn three, the "text under the box" is not text from the document at all: it is something the
model wrote about the picture on turn one. Feeding that back as the authority on numbers is exactly
the failure the rule was written to prevent, dressed up as compliance with it. So the two are
labelled differently all the way through, and a model-written description is explicitly marked as
not quotable and not authoritative on any figure.

## Why it is the same box everywhere[​](https://mindmax.me/#why-it-is-the-same-box-everywhere)

This is one action in every reader — PDFs, EPUB and Kindle books, PowerPoint slides, images, your
own Markdown notes — and the region adapts to what it was drawn on: a page in a PDF, a chapter in
a book, a slide number in a deck.

That uniformity is not tidiness for its own sake. The moment it exists in only some readers, you
stop reaching for it, because a gesture you have to check the availability of first is not a
gesture. It has to be reflexive to be worth anything.

## Keeping the answer[​](https://mindmax.me/#keeping-the-answer)

The last piece is the least clever and possibly the most useful. An explanation of a figure is
worth having next to your work, not buried in a conversation you will not scroll back through.

So a region and the answer it produced can be saved into your daily note in one step. When that
happens the picture is copied into your own folder — not referenced out of the conversation&#x27;s
storage — because a note that goes to a broken image after you tidy up old chats is worse than no
note at all.

## What is not claimed[​](https://mindmax.me/#what-is-not-claimed)

Handwriting and tiny axis labels are still where image reading goes wrong, and it goes wrong
confidently. The box does not fix that; it narrows it, by giving the model the text whenever text
exists and making it admit when there is none. And a figure that spans a page break is two regions,
not one.

The claim here is narrow and checkable: what you point at arrives with its context intact, the
numbers come from the document when the document has them, and the answer knows the way back.

[How region questions work →](https://mindmax.me/features/region-questions/)

**Tags:**

- [Notes](https://mindmax.me/blog/tags/notes/)
- [Accuracy](https://mindmax.me/blog/tags/accuracy/)
