

It’s usually some random thought that is in the right neighborhood, but not quite spot on. A human troubleshooter would straight up say that it’s impossible to tell you what the problem is, so you would need to narrow it down by testing a few things.
An LLM just says that you need to update drivers or whatever. If the problem is caused by something obscure (like this one), an LLM will never be able to figure it out. This kind of stuff apparently just doesn’t exist in the training data, so there’s no way for the model to extrapolate and reach the right conclusion. Instead, it will continuously interpolate with the data it has, and you’ll end up with an infinite list of wrong answers.
Sycophancy really doesn’t help either. If you have any ideas what might cause the problem, the LLM will cling to those, no matter how wrong you might be. Troubleshooting requires critical thinking and LLMs don’t seem to be very good at that.























The chances are very high. People always have technical problems, and they are also impatient enough to use an LLM. Why spend hours reading stackoverflow, and try solutions that aren’t exactly what you’re currently facing? That takes time, effort and improvisation skills.
Meanwhile, you could type a question to an LLM, and get an answer in a few seconds. In the best case scenario, you’ll get something useful out of it, but you could also start another wild goose chase. Humans are lazy, so they’ll fall into this trap very easily. It’s a gamble.