mapumbaa@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · 1 year agoGPT-OSS 20B and 120B Models on AMD Ryzen AI Processorswww.amd.comexternal-linkmessage-square11linkfedilinkarrow-up118arrow-down10
arrow-up118arrow-down1external-linkGPT-OSS 20B and 120B Models on AMD Ryzen AI Processorswww.amd.commapumbaa@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · 1 year agomessage-square11linkfedilink
minus-squareKissaki@programming.devlinkfedilinkEnglisharrow-up2·1 year agoFor those interested in the desktop-capable requirements: 16 GB GPU. 9070 XT? For lighting fast performance with the OpenAI GPT-OSS 20B model, users can use the AMD Radeon™ 9070 XT 16GB graphics card in a desktop system. Does it require that gen?
minus-squareafk_strats@lemmy.worldlinkfedilinkEnglisharrow-up2·1 year agoIf your video card has 16+ Gb of memory you will be able to run it with: NVIDIA cards GTX 10 series or later on ollama. Ollama is easy, but it leaves a lot of performance on the table. If you have less than 16 GB, you may be able to get good performance using llama.cpp or especially Ik_llama.cpp.
For those interested in the desktop-capable requirements: 16 GB GPU. 9070 XT?
Does it require that gen?
If your video card has 16+ Gb of memory you will be able to run it with: NVIDIA cards GTX 10 series or later on ollama.
Ollama is easy, but it leaves a lot of performance on the table.
If you have less than 16 GB, you may be able to get good performance using llama.cpp or especially Ik_llama.cpp.