We're getting customers that want to use LLMs to query databases and such. And I fully expect that to work well 95% of the time, but not always, while looking like it always works correctly. And then you can tell customers a hundred times that it's not 100% reliable, they'll forget.
So, at some point, that LLM will randomly run a complete non-sense query, returning data that's so wildly wrong that the customers notice. And precisely that is the moment when they'll realize, holy crap, this thing isn't always reliable?! It's been telling us inaccurate information 5% of the usages?! Why did no one inform us?!?!?!
And then we'll tell them that we did inform them and no, it cannot be fixed. Then the project will get cancelled and everyone lived happily ever after.
Or something. Can't wait to see it.