GPU acceleration

Answer speed is dominated by where the model runs. On a CPU, the first answer after a pause takes one to three minutes. On a mid-range NVIDIA card, seconds.

How it's selected

One setting in your .env:

COMPOSE_PROFILES=cpu     # or nvidia, or amd

The compose file ships three variants of the model container and starts exactly one of them. They share the same name, address and downloaded models, so switching between them keeps everything and re-downloads nothing.

The desktop app's guided setup detects this for you on first install. Everyone else picks it below.

Automatic detection

$vram = nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits 2>$null
$profile = if ($vram) { "nvidia" } else { "cpu" }
(Get-Content .env) -replace '^COMPOSE_PROFILES=.*$', "COMPOSE_PROFILES=$profile" |
  Set-Content -Encoding ascii .env
if ($vram) {
  Add-Content -Encoding ascii .env "CHRONICLER_GPU_VRAM_GB=$([math]::Floor($vram / 1024))"
}

curl -L -o detect-hardware.sh \
  https://github.com/ChroniclerLM/Releases/releases/latest/download/detect-hardware.sh
chmod +x detect-hardware.sh
./detect-hardware.sh

It checks the platform, whether containers can actually reach a GPU, and how much memory Docker has, then writes COMPOSE_PROFILES into the .env beside it - plus CHRONICLER_GPU_VRAM_GB on an NVIDIA host. It never touches your database password, and never picks a model: that stays with Admin → Model.

If it finds a card your containers can't reach, or Docker Desktop on the Hyper-V backend, it warns and settles for cpu rather than stopping. Read the warnings - they're the difference between a fast box and a slow one.

Useful flag: --env-file PATH (target a different .env).

The script needs bash - it runs on Linux, and on Windows from Git Bash or a WSL shell. If you can't run it, set the value by hand as below.

NVIDIA

nvidia-smi

nvidia-smi

docker run --rm --gpus all ollama/ollama:0.32.4 nvidia-smi

docker run --rm --gpus all ollama/ollama:0.32.4 nvidia-smi

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

COMPOSE_PROFILES=nvidia

docker compose up -d

docker compose up -d

AMD

Linux only, and best-effort. ROCm reaches the card through /dev/kfd and /dev/dri, which Docker Desktop on Windows cannot pass through.

ls /dev/kfd /dev/dri

If both exist:

COMPOSE_PROFILES=amd

docker compose up -d

This variant pulls a different, larger image (the ROCm runtime), so the first start after switching takes a while.

Verify it's actually being used

docker compose ps

docker compose ps

The running container should be ollama-nvidia or ollama-amd, not ollama-cpu. Then ask a question and watch:

nvidia-smi
docker compose logs chronicler-ollama | Select-Object -Last 30

nvidia-smi
docker compose logs chronicler-ollama | tail -30

The Ollama log says which backend it loaded the model onto. The honest test is the clock: if a warm answer still takes a minute, it's on the CPU.

Switching later

Edit COMPOSE_PROFILES, then docker compose up -d. Compose replaces the model container in place. Downloaded models live in a shared volume, so nothing is re-downloaded and no data is lost.

How the console learns about your GPU

Admin → Model probes hardware from inside the backend container, and that container is deliberately given no access to the graphics card - only the Ollama container gets it. Left alone, the console would report no GPU and grey out the GPU-class models on every machine, including yours.

CHRONICLER_GPU_VRAM_GB in your .env is what closes that gap. The detection script writes it, the desktop app's guided setup writes it, and the PowerShell snippet above writes it. The backend reads it and sizes the model list to the card you actually have.

Select-String "CHRONICLER_GPU_VRAM_GB" .env

grep CHRONICLER_GPU_VRAM_GB .env