floofloof@lemmy.ca to Technology@lemmy.worldEnglish · 4 months agoMotherboard sales are now collapsing amid unprecedented shortages fueled by AIwww.tomshardware.comexternal-linkmessage-square71linkfedilinkarrow-up1388arrow-down12file-textcross-posted to: technology@beehaw.orghackernews@lemmit.onlinepcgaming@lemmy.capcmasterrace@lemmit.online
arrow-up1386arrow-down1external-linkMotherboard sales are now collapsing amid unprecedented shortages fueled by AIwww.tomshardware.comfloofloof@lemmy.ca to Technology@lemmy.worldEnglish · 4 months agomessage-square71linkfedilinkfile-textcross-posted to: technology@beehaw.orghackernews@lemmit.onlinepcgaming@lemmy.capcmasterrace@lemmit.online
minus-squareSabinStargem@lemmy.todaylinkfedilinkEnglisharrow-up3·4 months agoYou can use something like KoboldCPP on Linux, which allows both RAM and VRAM combined to run a model. O’course, not as fast when compared to pure VRAM or the Mac approach, but it is an option. I use my 128gb RAM with some GPUs for running models.
You can use something like KoboldCPP on Linux, which allows both RAM and VRAM combined to run a model. O’course, not as fast when compared to pure VRAM or the Mac approach, but it is an option. I use my 128gb RAM with some GPUs for running models.