I have multiple models configured, but only one model is loaded at any given time

llama-server --port 8080 --host 0.0.0.0 \
  --models-preset /mnt/data/ai/models.ini \
  --models-max 1 \
  --tools all

look at models-max, since my hardware is potato, i dont want to load more than a model at a time, but if your hardware allows, you can load more than one.

from llama-server web interface, i can switch models based on what i need to get done (no need for llama swap).

hope this helps!

Nice write-up. Thanks for also including all the numbers. If I might ask: What is the thermal/throttling behaviour you mention? Is it still within the laptop’s thermal budget? Or does it reach throttling territory when doing inference on a long context window?

Thanks! Nowhere near throttling territory. The thermal ceiling is 95°C, and my sustained load is 72-78°C. There’s a 17-23°C safety margin.

The laptop never throttles. But the fans also don’t spin up properly during GPU-heavy inference unless I force it.

Why? ASUS’s embedded controller monitors CPU temperature and CPU package power for fan curves. It largely ignores GPU temperature.

I have a cpu only build of llama.cpp on a 32GB LPDDR5, 6400 MT/s. The laptop has an AMD Radeon 680M, but that is used for wayland, browser and GPU accelerated terminals.

Running llama-server, this is the performance: gemma-4 26B - 10.53 t/s

here is my llama-server command:

llama-server --port 8080 --host 0.0.0.0 --models-preset /mnt/data/ai/models.ini --ctx-size 8192 -ngl 0 --mlock --no-mmap

and here is the model.ini file

version = 1

[*]
; Global defaults - CPU-optimized
seed = -1
top-p = 0.95
top-k = 20
min-p = 0.05
presence-penalty = 0.0
repeat-penalty = 1.1
models-max = 2
jinja = true
batch-size = 256
ubatch-size = 128
threads = 8
threads-batch = 4
cpu-range = 0-7
cpu-strict = 1
kv-offload = false
poll = 25
poll-batch = 50
cpu-moe = true

[gemma-4-26b]
model = /mnt/data/models/daily/google_gemma-4-26B-A4B-it-Q4_K_M.gguf
temperature = 0.65
reasoning-budget = 384
repeat-penalty = 1.05

mcp-searxng - run your own searxng instance and connect to it

its glance app https://github.com/glanceapp/glance, a self hosted dashboard

i like the simplcity of waybar+mako+rofi much better than all-in-one-shells. Besides i believe in “do one thing best” policy of the smaller leaner apps

I use mkdocs https://github.com/squidfunk/mkdocs-material material for recipies, see screenshot

The custom widget can be found here

https://github.com/glanceapp/community-widgets/blob/main/widgets/cup-updates/README.md

It interfaces with https://github.com/sergi0g/cup

https://codeberg.org/ranjan/glance

Sure pls gimme some time to put it on codeberg

Thanks!

Its my daily on grapheneos

Thanks! But its a fork if Lite-XL called Pragtical - https://pragtical.dev/

Uses LuaJit instead of Lua as the differing factor from Lite-XL

Right!

dots posted in original post

If you are on hyperland already, only a couple of modules in waybar would need change. besides that everything remains the same.

i’ll post my dots soon maybe in a couple of hours

thanks!

Interesting!! I’ll look into this! Thanks!!

you could pick a random file from folader in a bash script… something like this:

RANDOM_FILE=$(find "$SOURCE_FOLDER" -type f | shuf -n 1)

I used 2 wallpaper applications to achieve this, swww and swaybg

1. For Niri, set layet rule

layer-rule {
    match namespace="^wallpaper$"
    place-within-backdrop true
}

2. Spawn at startup with niri swaybg -i .local/share/backgrounds/paint-blur.jpg &
3. Independent of niri use swww to set wallpaper swww img .local/share/backgrounds/paint.jpg

Hope this helps