I kinda get where he is coming for though. AI is being crammed into everything, and especially in things where they are not currently suited to be.

After learning about Machine learning, you kind realize that unlike "regular programs" that ML gives you "roughly what you want" answers. Approximations really. This is all fine and good for generating images for example, because minor details being off of what you wanted probably isn't too bad. A chat bot itself isn't wrong here, because there are many ways to say the same thing. The important thing is that there is a definite step after that where you evaluate the result. In simpler ML you can even figure out the specifics of the process, but for the most part we evaluate what the LLM said or if the image is accurate to our expectations. But we can't control or constrain the output to exactly our needs, because our restrictions largely are just input in a almost finished approximation engine.

The problem is, that companies take these approximation engines, put them in their product and consider their output fact. Like Ai chatbots doing customer support, and make up facts like the user that was told about rules that didn't exist for an airline, or the search engines that parrot jokes or harmful advice. Sure you and I might realize that these things come from a machine that doesn't actually think about it's answers, but others don't. And throwing a "*this might be wrong because its AI" on it is not an acceptable waiver of accountability.

Despite this, I use chatgpt and gemini a lot to help me program, they get a lot of things wrong but also do great. It's a great tool, exactly because I step in after the approximation step, review and decide. I'm aware of the limits. But putting these things in front of "users" without a review step means you are advertising that you are either unaware of this flaw, or just see the cost-benefit analysis and see that if noting else it'll generate interest during the hype.

There is a huge potential, but throwing AI into a situation where facts are needed when it's only making rough guesses, is the wrong way about it.