32 GB VRAM for less $1k sounds like a steal these days, and I’m sure it’s not getting cheaper any time soon.
Does anyone here use this GPU? Or any recent Arc Pros? I basically want someone to talk me out of driving to the nearest place that has it in stock and getting $1k poorer.
Vulkan helps with speed. Must benchmarks prove that out. Concurrency is a mixed bag. You can get some with llama.cpp bit vllm is concurrency king.
Just a couple of weeks ago llama.cpp released tensor parallelism which helps, but its still a experimental feature.
Unfortunately, I don’t know of any diffusion runners that work in vulkan. If someone has expertise, let me know!
In my experience Vulkan actually drops performance ever so slightly, but it also improves compatibility (especially on the chips AMD advertises as “AI” then promptly forgets to support via ROCm, like the gfx1101/02/03 family, gfx1150, etc.), which is why I recommend it - as you said, and I’ve just noted, AMD is famous about doing as little in their AI toolkit for the average user as possible, leading to very limited support on consumer hardware.