It's not 'lying' when they don't know the truth to begin with. They could be trying to answer accurately and it'd still be dangerous misinformation.
this post was submitted on 02 Jul 2025
116 points (98.3% liked)
Technology
72262 readers
4616 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
Not just health information. It is easy to make them LIE ABOUT EVERYTHING.
I sincerely hope people understand what LLMs are and what they're aren't. They're sophisticated search engines that aggregate results into natural language and refine results based on baked in prompts (in addition to what you provide), and if there are gaps, the LLM invents something to fill it.
If the model was trained on good data and the baked-in prompt is reasonable, you can get reasonable results. But even in the best case, there's still the chance that the LLM hallucinates something, that just how they work.
For most queries, I'm mostly looking for which search terms to use for checking original sources, or sometimes a reference to pull out something I already know, but am having trouble remembering (i.e. I will recognize the correct answer). For those use cases, it's pretty effective.
Don't use an LLM as a source of truth, use it as an aid for finding truth. Be careful out there!
No, it isn't. It's a fancy next word generator. It knows nothing, can verify nothing, shouldn't be used as a source for anything. It is a text generator that sounds confident and mostly human and that is it.
That depends on what you mean by "know." It generates text from a large bank of hopefully relevant data, and the relevance of the answer depends on how much overlap there is between your query and the data it was trained on. There are different models with different focuses, so pick your model based on what your query is like.
And yeah, one big issue is the confidence. If users are aware of its limitations, it's fine, I certainly wouldn't put my kids in front of one without training them on what it can and can't be relied on to do. It's a tool, so users need to know how it's intended to be used to get value from it.
My use case is distilling a broad idea into specific things to do a deeper search for, and I use traditional tools for that deeper search. For that it works really well.
Don't even use it as an aid for finding truth, it's just as likely, if not more, to give incorrect info
Why not? It's basically a search engine for whatever it was trained on. Yeah, it'll hallucinate sometimes, but if you're planning to verify anyway, it's pretty useful in quickly distilling ideas into concrete things to look up.
Yeah, I agree. It's a great starting place.
Recently I needed a piece of information that I couldn't find anywhere through a regular search. ChatGPT, Claude and Gemini all gave a similar answers, but it was only confirmed when I contacted the company directly which took about 3 business days to reply.
Meh. Google Gemini has given me great medical advice always couched carefully in “but check with your doctor.” and so on.
I was surprised too.