I wonder if we can extend the context length. It already fine-tuned with YaRN so we can’t get free extend with that method.

Ah. Sorry, good thing I attached the related link.

Everybody been rumoring about R2. So releasing this thing kinda unexpected

So something like

Previously the text talk about [last summary]
[The instruction prompt]...
[Current chunk/paragraphs]

There is new project which they share fine-tuned modernbert on some task. Here is the org https://huggingface.co/adaptive-classifier

Lots of developer choose to write in CUDA as ROCm support back then is a mess.

No, you can run sd, flux based model inside the koboldcpp. You can try it out using the original koboldcpp in google colab. It loads gguf model. Related discussion on Reddit: https://www.reddit.com/r/StableDiffusion/comments/1gsdygl/koboldcpp_now_supports_generating_images_locally/

Edit: Sorry, I kinda missed the point, maybe I’m sleepy when writing that comment. Yeah, I agree that LLM need big memory to run which is one of it’s downside. I remember someone doing comparison that API with token based pricing is cheaper that to run it locally. But, running image generation locally is cheaper than API with step+megapixel pricing.

Skywork downfall

There is koboldcp-rocm fork. Koboldcpp itself has basic image generation. https://github.com/YellowRoseCx/koboldcpp-rocm