Reminder: if you can put code in a chatbot and get it to find bugs, the devs can do it too. As such, even if your “LLM bug finding trip” works, it’s still useless, and a waste of everyone else’s time.
It is because everyone else’s computer is finding that same bug, and they’re all reporting it separately, and reduplicating efforts associated with trialling the bugs. It’s clearly more costly than having the dev themself fire a spare machine and have their computer alone find bugs.
The human will still need to check and understand the bug.
Particularly if a given bug is reported dozens of times in slight variations, someone will have to check for each of these reports whether it’s a bug already reported earlier. If they’re all checked by the same person, that person may quickly recognise “Okay, yeah, that’s the same thing I logged earlier” but if it’s multiple people, there’s just so much extra overhead associated with keeping track of what’s new and what’s “30th reported of same bug”.
The AI generating the report probably doesn’t check if the issue hss been found before. If the people subvmitting it also don’t, you end up with a load jof chaff by lazy people thinking they’re helpful when really they’re obstructing the efforts.
Reminder: if you can put code in a chatbot and get it to find bugs, the devs can do it too. As such, even if your “LLM bug finding trip” works, it’s still useless, and a waste of everyone else’s time.
Is it really a waste of time if the computer can find the bugs, and the dev can focus on fixing them instead of pulling double duty?
It is because everyone else’s computer is finding that same bug, and they’re all reporting it separately, and reduplicating efforts associated with trialling the bugs. It’s clearly more costly than having the dev themself fire a spare machine and have their computer alone find bugs.
The human will still need to check and understand the bug.
Particularly if a given bug is reported dozens of times in slight variations, someone will have to check for each of these reports whether it’s a bug already reported earlier. If they’re all checked by the same person, that person may quickly recognise “Okay, yeah, that’s the same thing I logged earlier” but if it’s multiple people, there’s just so much extra overhead associated with keeping track of what’s new and what’s “30th reported of same bug”.
The AI generating the report probably doesn’t check if the issue hss been found before. If the people subvmitting it also don’t, you end up with a load jof chaff by lazy people thinking they’re helpful when really they’re obstructing the efforts.