Torvalds: "100% of Their Code Is Written by Compilers" Too. The Linux creator argued that AI-written code claims are misleading — and warned that a wave of low-quality, AI-generated bug reports is quietly burning out the maintainers who keep open source running.
Exactly this. If you made an LLM that had deterministic code output then I’m all up for saying “AI is a compiler for human language”. But until then AI is most definitely not a compiler.
Um… yes. Matrix multiplication is deterministic, there’s nothing non-deterministic about an LLM, unless you artificially add randomness to them, like randomly selecting 1 of the top N ranked tokens.
The function is deterministic, agreed. The implementation is almost always not. Hardware floating point addition is not associative. So a GPU kernel that splits a reduction differently (ie interleaving it with anything else, like running your graphics, or sharing your work with other users on the same hardware) will produce different results over different runs even at temperature 0.0.
Determinism is almost always impossible when dealing with floating point on a multi-process/multi-user system.
Never seen a non-deterministic compiler though
Technically, LLMs (and most ML models) are deterministic with the same input and same seed.
I get what you mean though.
Or one that pretends compilation was a success when it wasn’t.
There is a parameter in llms called temperature. If you reduce it down to zero it will become deterministic. And probably even worse.
Yeah but the output would be crap. Just use the same prng seed and you’ll get reproducible output.
Exactly this. If you made an LLM that had deterministic code output then I’m all up for saying “AI is a compiler for human language”. But until then AI is most definitely not a compiler.
Setting the temperature parameter to 0.00 makes an LLM deterministic.
No. Floating point arithmetic and ordering of operations won’t make 0.0 deterministic.
Um… yes. Matrix multiplication is deterministic, there’s nothing non-deterministic about an LLM, unless you artificially add randomness to them, like randomly selecting 1 of the top N ranked tokens.
The function is deterministic, agreed. The implementation is almost always not. Hardware floating point addition is not associative. So a GPU kernel that splits a reduction differently (ie interleaving it with anything else, like running your graphics, or sharing your work with other users on the same hardware) will produce different results over different runs even at temperature 0.0.
Determinism is almost always impossible when dealing with floating point on a multi-process/multi-user system.
There are attempts to create batch invariant language models (https://github.com/thinking-machines-lab/batch_invariant_ops) but all the major ones are not.