• 2 Posts
  • 23 Comments
Joined 7 days ago
Cake day: October 31st, 2026
  • Spite_By_Design_LLC@lemmy.dbzer0.comOPtoLocalLLaMA@sh.itjust.works•British fintech Revolut slashes AI coding costs 10x by cutting out Claude and going self-hostedEnglish
    1·
    19 hours ago

    Absolutely. And given my druthers, I’d grab the 128. The issue is, it’s about triple the cost locally vs the 64.

    I’m going to have to benchmark QFN or hell, even Qwen3.6-35B to see if they are capable for the tasks I want in the constrained environment (at sea, solar powered).

    I won’t go into the minutae, but tasks like this via Signal K MCP- using wake word, STT and TTS

    Me: “Hey Boat, how far are we from Palau?”

    Boat: “We are 420 nautical miles from Palau, with approximately 510 nautical miles remaining on our planned route.”

    Me: “What is the weather like?” Boat: “The wind is south-westerly at 12 knots. The barometer reads 1016 hPa. Our latest forecast indicates stronger winds tomorrow.”

    Me: “And the day after that?” Boat: “The latest 7 day forecast indicates increasing winds tomorrow afternoon. Would you like the details?”

    Me: “Yes. And given our present course, should we consider another anchorage?”

    Boat: “Possibly. Let me compare the forecast with nearby anchorages and their exposure.”

    IOW, the MCP system communicating and tallying actual on board sensors to help with navigation, decision making etc.

    I probably won’t need to vibecode the next $1M ARR SAAS while pottering around the Western Pacific but I will need something small, smart and fast that can work onboard within constrained power envelope (say… 3kwh per day across all systems).

    For something that can WoL (I use a Raspberry pi as an always on SBC), answer my question and then piss off without gobbling power, the Strix looks promising.

  • Spite_By_Design_LLC@lemmy.dbzer0.comOPtoLocalLLaMA@sh.itjust.works•British fintech Revolut slashes AI coding costs 10x by cutting out Claude and going self-hostedEnglish
    2·
    2 days ago

    Agreed. Economies of scale are about the only way this makes sense.

    I was just working thru the use case for purchasing a 64gb Strix (vs the typical coding I might do using OR or Open Code Go). The ROI is somewhere in the order of 7-10 years for me. So, on a purely dollar cost basis, local inference is $$$. OTOH, from a sovereignty and “own your own code” basis…there’s something to be said for it.

    Probably I will run a few tests using OR with a few outstanding tickets of mine, to kick the tyres of Qwen3.8-27B. I’ve tried Qwen3.6-35B and frankly it’s…not very good at code gen or patching.

    To boot, code gen isn’t the only thing I use LLM for, so I suppose it depends on how bloody minded I want to be about future use case (I do actually have one).

  • Spite_By_Design_LLC@lemmy.dbzer0.comOPtoLocalLLaMA@sh.itjust.works•British fintech Revolut slashes AI coding costs 10x by cutting out Claude and going self-hostedEnglish
    1·
    2 days ago

    BRB, just getting some scratch lotto tickets.

    14.3 kw. That’s fucking obscene.

    And by all rights, the Fintech datacentre is what…small? medium?

    GLM again:

    One DGX B200 flat-out: 14.3 kW × 24h = ~343 kWh/day ≈ what a typical UK house uses in a month

    Datacentre size: a full rack of AI gear like this is ~50–100kW. Classification-wise:

    Your average office server closet: 5–20 kW
Small datacentre: <1 MW
Mid-size / enterprise: 1–10 MW
Hyperscale (AWS/Google/Microsoft): 50–500+ MW per campus

    Revolut’s 56 GPUs ≈ one dense rack or two — well under 1 MW, so firmly “small,” and that’s exactly why they colocate/rent space (“a US-based service”) rather than owning a facility. Self-hosting the GPUs doesn’t mean self-hosting the building. You don’t put 100kW of fridges in your London office — you rent rack space in a proper colo where the power and cooling already exist.

    Enjoy the lotto. If you win, you can buy an eighth of a DGX.

    Fuuuuuuuuuck

  • Spite_By_Design_LLC@lemmy.dbzer0.comOPtoLocalLLaMA@sh.itjust.works•British fintech Revolut slashes AI coding costs 10x by cutting out Claude and going self-hostedEnglish
    2·
    2 days ago

    In the spirit of things, I asked GLM5.3 what that set up actually looks like. Ouch.

    Fetched that page for real. Specs (from gpuworld’s DGX B200 datasheet page):

    Dimensions: 17.5" W × 87.5" D × 42.9" H (cm: 44.5 × 222.3 × 109.0)
Weight: 275 lb (124.7 kg)
Rack height: 10U equivalent
Power: up to 14.3 kW max per system

    In real-world terms:

    Height: 10U = 10 rack units = 10 × 1.75" = 17.5" tall — about the height of a bar fridge standing on its side. But that's the chassis; you'll also want ~2U of clearance for airflow/management, so plan for ~1.5U of breathing room above it in a rack.
Depth: 87.5cm = just under a metre — so it'll stick out of a standard 1m-deep server rack unless you've got deep racks (or you let the back panel hang open). Standard 42U racks are typically 1.1–1.2m deep to handle this.
Weight: 275 lb / 125kg. That's a hefty man's weight — two people minimum to lift, ideally four or a rack lift. Standard datacentre floor loading is usually fine at this density but worth checking if you're stacking multiple in a rack.

    Rack math for 7 units: 70U of just these systems — so a single 42U rack can’t hold them all (you’d need 2 racks), and at ~14kW each you’re looking at ~100kW total draw — that’s serious power/cooling, way beyond a standard office closet.

    Short version: think “fridge-sized box, half a tonne with rails, eats 14kW” — not a pizza box.