Skip to main content

Ollama and LM Studio for your documents — what a local model can and cannot carry

· 4 min read
The MindMax team
The MindMax team
Building MindMax

A local model through Ollama or LM Studio handles short, contained document work well — explaining a passage, answering a question about an open paper, tidying a note — and that work never leaves your machine. It struggles with long jobs that need a large context and reliable tool use: reading a whole book chapter by chapter, translating hundreds of pages, drafting something book-length.

That split is worth knowing before you start, rather than four hours into a job that will not finish.

Why people want this​

The reason is usually one sentence: nothing may leave the machine. A contract under NDA, patient notes, an unpublished manuscript, a client's financials. For that material, "the text goes to a provider with a good privacy policy" is not the same answer, and it is right to treat it differently.

A model running locally is the only configuration where the honest answer is "nothing is sent anywhere". Bringing your own cloud API key removes one party from the path; it does not remove the path.

What a local model is good at​

Laptop-sized models have improved fast, and for a lot of everyday document work they are genuinely fine:

  • Explaining a selected passage — the text is short, the question is local.
  • Questions about one open document that fits comfortably in the model's context.
  • Tidying, rephrasing and summarising a note you wrote yourself.
  • Quick edits where you will read every change anyway.

For these, speed is the main cost, and on a recent machine it is usually acceptable.

Where it struggles​

Three properties separate the big jobs from the small ones, and laptop-sized models are weakest at all three:

  1. Long context. Reading a book well means holding a lot of text at once. Local models often run with a much smaller context than their cloud counterparts, or slow down sharply as it fills.
  2. Dependable tool use. Navigating a book — open the contents, pick the chapters, read them, go back for another — is a sequence of tool calls. A model that gets one call slightly wrong derails the whole sequence.
  3. Staying consistent over hours. A three-hundred-page translation or a long draft is hundreds of calls that all have to follow the same instructions. Small models drift.

So: an AI reading pass over a whole book, whole-book translation and book-length drafting are the jobs we would not start on a local model. It will be slow at them, or it will not finish.

Reading images is a separate question​

Asking about a boxed chart, recognising a scanned page or reading a photographed whiteboard needs a model that takes pictures. Many popular local models are text-only. If yours is, those features have nothing to run on until you point that job at a model that reads images.

Splitting the work​

The practical setup for most people is not all-or-nothing. MindMax groups model work into tiers — quick work, everyday work, the passes where being right matters, and reading images — and you choose which model sits in each, with per-task overrides.

  • If nothing may leave the machine, point every tier at the local model and accept that the long jobs are out of reach. Keep those documents in their own folder so you always know which rules apply.
  • If some material is sensitive and some is not, keep the local model for the sensitive folder's everyday questions and use a cloud model for long jobs on everything else.
  • If you just want to save money, a local model for quick work is a reasonable place to start. Measure before assuming it is cheaper overall; your time is part of the bill.

Local providers need no key. Requests go to an endpoint on your own machine, and nothing about them reaches us.

What is not claimed​

We have not published benchmarks for specific local models, and capability changes month to month. The claim here is about the shape of the work: short and contained is fine, long and multi-step is where a laptop-sized model runs out. Try the job you actually need on a short document first.

Setting it up is in use your own API key, and the full picture — including where your own key is and is not the same as privacy — is on bring your own key. The reasoning behind making every feature work on your own key is in the subscription is a convenience, not a gate.