347
this post was submitted on 27 Aug 2025
347 points (97.0% liked)
Technology
74519 readers
4626 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I've seen maybe 4 articles like this vs the hundreds of millions that use it everyday. I think the ratio of suicide vs legitimate use of rope is higher actually. And no, being told bad things by a jailbroken chatbot is not the same as being shot.
Have you seen the ai girlfriends/boyfriends communities? I genuinely think the rate of chatgpt induced psychosis is really high, even if it doesn't lead to death
i didn't say they were the same, you put those words in my mouth. I put them both in the category of things that need regulation in a way that rope does not. are you seriously of the opinion that it is fine and good that people are using their ai chatbots for mental healthcare? are you going to pretend to me that it's actually good and normal for a human psychology to have every whim or fantasy unceasingly flattered?
My whole point since the beginning is that this is dumb, hence my comment when you essentially said shooting projectiles and saying bad things were the same. Call me when someone shoots up a school with AI. Guns and AI are clearly not in the same category.
And yes, I think people should be able to talk to their chatbot about their issues and problems. It's not a good idea to treat it as a therapist but it's a free country. The only solution would be massive censorship and banning local open source AI, when it's very censored already (hence the need for jailbreak to have it say anything sexual, violent or on the subject of suicide).
Think for a second about what you are asking and what it implies.
no i didnt say that because it's a stupid fucking thing to say. i dont need your hand up my ass flapping my mouth while im speaking, thanks.
How about I call you when a person kills themself and writes their fucking suicide note with chatgpt's enthusiastic help, fucknozzle? Is your brain so rotted that you forgot the context window of this conversation already?
You can't defend your position because it's emotional exaggeration. Now you're lashing out and being insulting.
My whole point is that they aren't the same and you keep saying "let's treat them as if they were", then you use it in comparisons and act like a child when I point out how silly that is.
Clarify what you mean. Take the gun out of the conversation and stop bringing it up. Stop being disingenuous. Don't be a baby.
you want me to explain it differently, and I will. that's a very reasonable request.
i think we should regulate things that can be shown to be dangerous to indiviiduals or society as a whole. I will take your rope example as not dangerous in that way and leave it unexamined, assuming you agree. compare to guns. guns are dangerous and you seem to agree with this too. rope is different from a gun, but both can be used to kill people. why don't we regulate rope? in a nutshell, because it takes a hell of a lot of effort to hurt or kill someone with rope. compare to a gun. the amount of effort required to kill a person, many people, with a modern firearm is a physical triviality comparable to brushing your teeth or changing your clothes. guns can be harmful without even trying, but you have to go out of your way to hurt someone with rope.
compare with the current unregulated implementation of chatbots, as in the case of this child's suicide. a technology which can calmly sit with you and convince you that your suicide is a beautiful expression of individuality or whatever sycophantic bullshit that desperate child read.
here, let's remind ourselves of some of the details presented in the article. This will no doubt be a refresher for you.
so, according to this lawsuit, a child was taught to circumvent chatgpt's safety measures by chatgpt itself, encouraged to commit suicide, and this all happened despite the fact that the model was specifically trained not to do this. this happened despite the large amount of effort that was put in to avoiding something like this.
that this is even a possibility means we do not have the control over this technology it might otherwise appear that we do. Uncontrollable technology is dangerous. Dangerous technology should be regulated. thanks for coming to my ted talk.
for more information about AI safety, check out robert miles.