this post was submitted on 27 Aug 2025
487 points (96.4% liked)

Technology

74924 readers
3168 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

The makers of ChatGPT are changing the way it responds to users who show mental and emotional distress after legal action from the family of 16-year-old Adam Raine, who killed himself after months of conversations with the chatbot.

Open AI admitted its systems could “fall short” and said it would install “stronger guardrails around sensitive content and risky behaviors” for users under 18.

The $500bn (£372bn) San Francisco AI company said it would also introduce parental controls to allow parents “options to gain more insight into, and shape, how their teens use ChatGPT”, but has yet to provide details about how these would work.

Adam, from California, killed himself in April after what his family’s lawyer called “months of encouragement from ChatGPT”. The teenager’s family is suing Open AI and its chief executive and co-founder, Sam Altman, alleging that the version of ChatGPT at that time, known as 4o, was “rushed to market … despite clear safety issues”.

(page 2) 50 comments
sorted by: hot top controversial new old
[–] Occhioverde@feddit.it 3 points 1 week ago* (last edited 1 week ago) (2 children)

I think we all agree on the fact that OpenAI isn't exactly the most ethical corporation on this planet (to use a gentle euphemism), but you can't blame a machine for doing something that it doesn't even understand.

Sure, you can call for the creation of more "guardrails", but they will always fall short: until LLMs are actually able to understand what they're talking about, what you're asking them and the whole context around it, there will always be a way to claim that you are just playing, doing worldbuilding or whatever, just as this kid did.

What I find really unsettling from both this discussion and the one around the whole age verification thing, is that people are calling for techinical solutions to social problems, an approach that always failed miserably; what we should call for is for parents to actually talk to their children and spend some time with them, valuing their emotions and problems (however insignificant they might appear to a grown-up) in order to, you know, at least be able to tell if their kid is contemplating suicide.

[–] LillyPip@lemmy.ca 1 points 1 week ago* (last edited 1 week ago) (1 children)

but you can't blame a machine for doing something that it doesn't even understand.

But you can blame the creators and sellers of that machine for operating unethically.

If I build and sell a coffee maker that sometimes malfunctions and kills people, I’ll be sued into oblivion, and my coffee maker will be removed from the market. You don’t blame the coffee maker, but you absolutely hold the creator accountable.

[–] Occhioverde@feddit.it 1 points 1 week ago* (last edited 1 week ago) (1 children)

Yes and no. The example you made is of a defective device, not of an "unethical" one - though I understand how you are trying to say that they sold a malfunctioning product without telling anyone.

For LLMs, however, we know damn well that they shouldn't be used as a therapist or as a digital friend to ask for advice; they are no more than a powerful search engine.

An example that is more in line with the situation we're analyzing is a kid that stabs itself with a knife after his parents left him playing with one; are you sure you want to sue the company that made the knife in that scenario?

load more comments (1 replies)
[–] pelespirit@sh.itjust.works -1 points 1 week ago (2 children)

What I find really unsettling from both this discussion and the one around the whole age verification thing

These are not the same thing.

[–] Matthew@midwest.social 2 points 1 week ago

Isn't that probably why they differentiated them in the sentence you quoted?

load more comments (1 replies)
[–] chrischryse@lemmy.world 1 points 1 week ago (4 children)

OpenAI shouldn’t be responsible. The kid was probing ChatGPT with specifics. It’s like poking someone who repeatedly told you to stop and your family getting mad at the person for kicking your ass bad.

So i don’t feel bad, plus people are using this as their own therapist if you aren’t gonna get actual help and want to rely on a bot then good luck.

load more comments (4 replies)
load more comments
view more: ‹ prev next ›