LLMs suck at writing code, but they are absolutely amazing at reading code and finding bugs. I am firmly against the usage of LLMs to write anything, but they have a place as a diagnostics tool when verified by a human.
Yeah I’ve been using LLM’s in Rider for over a decade. I don’t have a problem with LLM’s. I do have a problem with how they’re currently used, and how people keep trying to use them to replace their own thinking.
I think they have a place in the coding scene, a limited niche place, but a place none the less. They just aren’t a replacement for software engineers. Architecture and intention are the big differences to me. An LLM cannot understand intention, it just makes statistical guesses that are often wrong.
IMO: Specifically, they’re bad at architecture and refactoring a small project into a large project. You have to jump through some hoops to make it craft something that needs more than 8m of context ram. If you can manage orchestration and multi-agents that don’t need to know each others context, you can start to pull off bigger stuff, but it’s not a forgone conclusion that it’ll be fine. The worst output comes from it getting stuck on something and trying less likely answers successively until it works. You really have to watch for it to struggle and at the very least stop and start to try some new randoms.
Also, anything other than Claude-code with some really well-done project definitions is a waste of time.
There are already several large projects that have hundreds of thousands of users and those projects were almost completely vibe coded. It’s almost the entire retro “recomp” scene now.
At my company we are currently moving to implement AI for exactly that. A cost effective review buddy.
Various developers have tried to use it for coding and I think apart for one time scripts or getting an initial structure generated, noone is convinced of AI.
More than anything, it depends on the developer using it. In my experience, most are just slop shovelers that don’t read or review what Claude does. Then it falls on the PR reviewer (me) to be an asshole: Did you review your own code? Do you understand what this part over here does? Why did you choose this method versus alternatives?
Unless you tell Claude/Opus otherwise, it’s not going to take the initiative to refactor things, tidy up parts it didn’t touch, or hell, it won’t even spellcheck things. Can you really call it a language model if it outputs misspelled words? Its unit tests are great boilerplate but never comprehensive enough for me.
Claude is the developer that puts zero pride or effort in their work beyond the bare minimum to not get fired.
Yes I agree, and honestly code review is now the key skill.
My company recently started exploring “AI interviewing”, to test candidates on “AI proficiency”, and the questions are all around prompt engineering (is that even a thing anymore?) and stuff. Like no, make sure they can read the fucking code!
I’m not terribly concerned if people write code and test it with AI, so long as they verify the results themselves.
I recently saw someone try to add a PR to something on github with thousands of lines of changes and then they got mad when people didn’t want to do the free labour verifying it. That is a bullshit use of AI
Refactoring and consolidating a big preexisting codebase. I try to introduce new concepts on a semantic level and it tries to give me interfaces and abstractions that would obstruct my work in the long run.
If we ever find a way to deal with the whole “burning the planet” issue, their best use case is to find and match patterns, not to imitate them.
The idea of using a language model to process search engine input isn’t the dumbest part about Gemini and plenty of people report good results finding information easier and quicker with ChatGPT. In those cases, the deviation from rigid keywords is desirable because it can match results with related words rather than literal word-matching. Google Search already did a decent job at that (before the enshittification ran rampant), which more complex language models could improve even further.
The landmine is in their reproduction of those results, where the generated “summary” is the equivalent of a cunning bullshitter that convincingly sounds like he understood the topic but actually has no clue and just delivers a best guess. That’s where the deviation becomes a risk of misinformation or introducing bugs.
They should narrow things down by finding the likely answers where that matters, not produce more stuff that humans will have to double-check.
LLMs suck at writing code, but they are absolutely amazing at reading code and finding bugs. I am firmly against the usage of LLMs to write anything, but they have a place as a diagnostics tool when verified by a human.
Line completion is pretty damn handy, it’s when you start asking for whole functions that things go downhill
Yeah I’ve been using LLM’s in Rider for over a decade. I don’t have a problem with LLM’s. I do have a problem with how they’re currently used, and how people keep trying to use them to replace their own thinking.
I think they have a place in the coding scene, a limited niche place, but a place none the less. They just aren’t a replacement for software engineers. Architecture and intention are the big differences to me. An LLM cannot understand intention, it just makes statistical guesses that are often wrong.
IMO: Specifically, they’re bad at architecture and refactoring a small project into a large project. You have to jump through some hoops to make it craft something that needs more than 8m of context ram. If you can manage orchestration and multi-agents that don’t need to know each others context, you can start to pull off bigger stuff, but it’s not a forgone conclusion that it’ll be fine. The worst output comes from it getting stuck on something and trying less likely answers successively until it works. You really have to watch for it to struggle and at the very least stop and start to try some new randoms.
Also, anything other than Claude-code with some really well-done project definitions is a waste of time.
There are already several large projects that have hundreds of thousands of users and those projects were almost completely vibe coded. It’s almost the entire retro “recomp” scene now.
At my company we are currently moving to implement AI for exactly that. A cost effective review buddy.
Various developers have tried to use it for coding and I think apart for one time scripts or getting an initial structure generated, noone is convinced of AI.
Opus-class and higher models are actually great at writing code, but maybe it’s language dependent? I primarily work in TypeScript.
More than anything, it depends on the developer using it. In my experience, most are just slop shovelers that don’t read or review what Claude does. Then it falls on the PR reviewer (me) to be an asshole: Did you review your own code? Do you understand what this part over here does? Why did you choose this method versus alternatives?
Unless you tell Claude/Opus otherwise, it’s not going to take the initiative to refactor things, tidy up parts it didn’t touch, or hell, it won’t even spellcheck things. Can you really call it a language model if it outputs misspelled words? Its unit tests are great boilerplate but never comprehensive enough for me.
Claude is the developer that puts zero pride or effort in their work beyond the bare minimum to not get fired.
Yes I agree, and honestly code review is now the key skill.
My company recently started exploring “AI interviewing”, to test candidates on “AI proficiency”, and the questions are all around prompt engineering (is that even a thing anymore?) and stuff. Like no, make sure they can read the fucking code!
I’m not terribly concerned if people write code and test it with AI, so long as they verify the results themselves.
I recently saw someone try to add a PR to something on github with thousands of lines of changes and then they got mad when people didn’t want to do the free labour verifying it. That is a bullshit use of AI
Opus in my experience sucks for writing code.
What are you trying to have it do? I work in a very large and complex codebase, and it honestly scares me sometimes.
Refactoring and consolidating a big preexisting codebase. I try to introduce new concepts on a semantic level and it tries to give me interfaces and abstractions that would obstruct my work in the long run.
The lemmysphere will never agree
I used to be of that mind as well. But the newer models are just too good to deny it any longer.
Same. There’s an overwhelmingly negative opinion here.
If we ever find a way to deal with the whole “burning the planet” issue, their best use case is to find and match patterns, not to imitate them.
The idea of using a language model to process search engine input isn’t the dumbest part about Gemini and plenty of people report good results finding information easier and quicker with ChatGPT. In those cases, the deviation from rigid keywords is desirable because it can match results with related words rather than literal word-matching. Google Search already did a decent job at that (before the enshittification ran rampant), which more complex language models could improve even further.
The landmine is in their reproduction of those results, where the generated “summary” is the equivalent of a cunning bullshitter that convincingly sounds like he understood the topic but actually has no clue and just delivers a best guess. That’s where the deviation becomes a risk of misinformation or introducing bugs.
They should narrow things down by finding the likely answers where that matters, not produce more stuff that humans will have to double-check.
So far.