- cross-posted to:
- programming@programming.dev
- cross-posted to:
- programming@programming.dev
You must log in or # to comment.
Well, unlike the compiler I don’t trust the AI enough not to re read the result.
For now it’s the pair programing buddy that helps me debug. There are stuff where it’s definitely more efficient than a human is.
I understand the spirit of what he means, but I still don’t know how an LLM can be used efficiently as a precise tool like this. If I can describe a problem narrowly enough to guide an LLM to give a useful output, I’ve already solved it myself and may as well just type it out.
Exactly same thinking. I can usually type it out faster than AI (when taking into account latency, context building, prompt writing, prompt fixing, AI hallucination review) if I know what exactly needs to be done.
When you’re out of your depth, it shines in providing beautifully confident and a botanical garden of a code piece that wil most likely be broken and break in new and exciting ways. Great for incompetent sycophantically-challenged managers that forgot how to code, or never actually had any experience.
Large-scale code transformations, taking into account the bigger picture of the repository are most likely hallucination free (not generative, just transformative, as per the actual LLM model) and a very exciting use-case.
Also it’s currently a very nice pragmatic tool for checking for any mistakes, because it can connect the larger context of the repository quickly to the diff. Terrible if done by a manager without understanding of copilot and workflow, but a great tool if done through claude and cli as a pre-commit step with just quick checklists. Makes plenty of mistakes, but allows to catch your own big mistakes nicely quickly.
Never seen a non-deterministic compiler though
Technically, LLMs (and most ML models) are deterministic with the same input and same seed.
I get what you mean though.
Or one that pretends compilation was a success when it wasn’t.
Exactly this. If you made an LLM that had deterministic code output then I’m all up for saying “AI is a compiler for human language”. But until then AI is most definitely not a compiler.
Setting the temperature parameter to 0.00 makes an LLM deterministic.
No. Floating point arithmetic and ordering of operations won’t make 0.0 deterministic.
Um… yes. Matrix multiplication is deterministic, there’s nothing non-deterministic about an LLM, unless you artificially add randomness to them, like randomly selecting 1 of the top N ranked tokens.
The function is deterministic, agreed. The implementation is almost always not. Hardware floating point addition is not associative. So a GPU kernel that splits a reduction differently (ie interleaving it with anything else, like running your graphics, or sharing your work with other users on the same hardware) will produce different results over different runs even at temperature 0.0.
Determinism is almost always impossible when dealing with floating point on a multi-process/multi-user system.
There are attempts to create batch invariant language models (https://github.com/thinking-machines-lab/batch_invariant_ops) but all the major ones are not.
Linus seems to be taking a perfectly pragmatic approach, given that AI is not going away short of WW3.
I can imagine some individual sub-system maintainers introducing various AI-roadblocks though.
I also expect this to be increasingly addressed (in general) with model & tooling improvements, giving more weight to higher quality reports and MRs, and more respect for project rules and processes. A mix of soft and hard gates, CLAs, improved early automated bug report & patch reviews and other CI gates.
If there’s anybody I trust to use LLMs in a responsible way, (to whatever degree that’s even possible) it would be Linus Torvalds. He has some of the highest standards for software development in the world, and the same can be said for the core Linux kernel development team.
The general problem though, is that the people who are the least qualified to do software development are very often the people who are the most likely to use LLMs to try to slop their way to the top.
I think that the combination of cost increases for inference, inherent limitations in model integrity, and the economic destruction that will ensue when these fraudster CEOs have finally squeezed every last drop of blood from every last stone they can find, will leave “AI” in a similar place as blockchain/crypto/NFTs.
There will be a relatively niche market for software engineers and related tech workers; local LLM boxes, some subscription-based models that companies pay for with their licensing packages, hobbyists, and some minimal integrations into software suites for tools like background removal, lighting enhancement, etc.
There will be no Singularity, no AGI, no all-knowing Machine God.
I wish more would listen.
Many seem to interpret the headline as “Torvalds is an AI Bro now,” when its more like him yelling “If you all don’t treat these things as tools, shit is going to hit the fan.”
Entryism. Promote a “rational adaptation” scheme, reject “slop”, embrace “gems”, try to use it as a “helping tool”.
The reality is much grimmer: Some vibe coders lie about them only using it as a tool, them “reading the output” is just getting it compiled and maybe pass some (AI generated) tests, and overly trust the genAI over classical search engines. People using the tools show some degree of AI psychosis, such as trying to “prompt” flesh and blood people (“I’ve read on a blog that…” is often used to make chatbots agree with you, and is also used towards humans by those who talk to chatbots a bit too much).
After seeing what happened to windows because of this “tool” I think many of us are rightfully skeptical of what will happen to this community.
It happened before, why are we okay with it happening again?
Microsoft doesn’t treat it as a tool; that’s the problem.
…Which isn’t surprising. Not only because Microsoft is Microsoft, but they own like a third of OpenAI, so they have a vested interest in perpetuating the AI hype.
Microsoft has far more people relying on their products to work, so you’d think they’d exercise far more diligence in making things work…
Well… Microsoft is Microsoft.
This is what they do. They simply cannot help themselves.
It honestly makes me distrust the whole move because you would genuinely think that if the world depends on your stuff to work you would make it work, AI or not.
We’re still talking about Microsoft, right?
They have a looooong history of enshittification and footgunning that predates “AI” by decades, no matter how much the world has depended on their stuff. So no, I suppose I wouldn’t think that, as that would break decades of precedence for MS. But I suppose I never trusted them in the first place.
It’s strange to me because well, I was a 20+ years windows veteran who managed to never have issues on any of my windows installs prior to the last couple years when the ai thing went full swing. maybe part of that was me, because I was inattentive and reckless, but it really just became too much to ignore. i have a genuinely hard time with the idea of essentially retconning the way things always used to Just Work for me in my Windows days. maybe it was bad when I was there and I just didn’t notice. but blegh.
Even Linus is conflating “doing more” with “being more productive”. A car factory could, if they wanted, pump 10x more cars out the door. But qualiry would decrease these would be full of defects, so in the end you weren’t more productive
I don’t care if there’s more PRs. It’s just and illusion of productivity
Productivity = Doing more things Efficiency = Doing the right things
I wrote a whole thesis on the subject AMA
Wouldnt productivity have two knobs? Efficiency and energy/effort input?
Also efficiency sometimes goes against “right”? If you need resiliancy you then “right” migh include redundancy. In software, that could also be guardrails like memory safety that necissarily take some extra abstraction to implement and a processing penalty?
