Models
The model catalog
Every model here also reads images, because the same model answers questions and does AI-assisted OCR on scanned pages. You only ever run one.
| Model | Needs | Download | Good for |
|---|---|---|---|
gemma4:e2b | 8 GB RAM, no GPU | ~7 GB | The default. Fastest chat and OCR, strong French and Arabic. Weaker on dense tables. |
gemma4:e4b | 16 GB RAM, no GPU | ~10 GB | Better reasoning and OCR than E2B, noticeably slower on a CPU. |
qwen2.5vl:7b | 8 GB VRAM | ~6 GB | Best table and document OCR at this size. |
gemma4:12b | 10 GB VRAM, 16 GB RAM | ~8 GB | Long context, solid reasoning and dense-table OCR. |
gemma4:26b | 20 GB VRAM, 32 GB RAM | ~18 GB | Best answer quality on long documents. |
gemma4:31b | 24 GB VRAM, 32 GB RAM | ~20 GB | Largest tier, slowest per answer. |
The first two run without a graphics card. The rest need one.
Changing the model
Admin → Model shows what the server detected, what it recommends, and the catalog. Anything your hardware can't run is greyed out.
Pick one and choose "Use this model"
The download starts immediately, with a progress bar. It's a multi-gigabyte download — leave it alone.
Wait for it to finish
The new model only becomes active once its download completes, so chat keeps working on the old one meanwhile.
The old one is removed
Once the switch is done, the superseded model is deleted to reclaim the disk.
No restart is needed, and your documents and index are untouched — only the component that writes the answers changed.
Choosing a GPU model by hand
The hardware probe runs inside the backend container, which has no access to the graphics card. It therefore reports no GPU, and every GPU-class model above is greyed out even on a machine that has one.
Until that's fixed, set the model on the server instead:
Download the model into the stack
docker exec chronicler-ollama ollama pull gemma4:12b
Tell the backend to use it
In your docker-compose.yml, under the backend service:
environment:
OLLAMA_GENERATION_MODEL: gemma4:12b
Keep the existing DATABASE_URL and POSTGRES_* entries as they are — add this
line to them.
Restart
docker compose up -d
The embedding model is fixed
Answers come from the generation model above, but finding the right passages is done by a separate embedding model. That one cannot be changed: every document you've indexed was indexed with it, and swapping it would silently invalidate the lot. It ships inside the image, so it never needs downloading and works with no internet at all.
Re-indexing
Admin → Model → Reindex all rebuilds the search index for every document.
You need it rarely — after a restore from an older backup, or if search results look wrong. It's heavy: expect the server to be busy for a long time on a large library, and run it outside working hours.
Choosing well
- On a CPU, stay small.
gemma4:e2banswering in 40 seconds beatsgemma4:e4banswering in four minutes. - On a GPU, use the largest model that fits comfortably in VRAM. One that almost fits spills into system memory and gets dramatically slower.
- For scanned documents and tables,
qwen2.5vl:7band the 12B tier and above read structured pages much better. - Test with your own documents. Benchmarks don't know what your contracts look like.