212
Certain names make ChatGPT grind to a halt, and we know why
(arstechnica.com)
good ol leetspeak
We truly live in the best timeline.
I think your typo helped it get past the filter, not the leetspeak. It said it didn’t know, and hen when you said “look it up,” the search results autocorrected and that’s how you got past the filter.
Side note: I had to insist on leetspeak that many times at the end or it would go "brian...[error message]"
I shall have to try again
I miss IRC.
We still out here dowg.
I love that it started devolving into a working-class British accent in the end, for no apparent reason
It's still a text predictor. Your average corporate representative isn't going to use leetspeak, and so the probable next set of words won't be as similar.
His name is Brian Hood not Brain Hood, or am I missing the joke, in that case whoosh I guess
Maybe it was a way to get the engine to say it doesn’t know “brain” hood, and when they asked it to look it up, their hits autocorrected to “Brian,” and that’s how they got the information past the filter. Which would be incredibly clever, and it’s I believe how it actually got past it, not the leetspeak.
Nop, it's the leetspeak. That trick has worked great for me, I don't know why chatgpt hasn't patched it yet. Google figured that out back when their servers held 40gb and were build out of legos
I posted another example above using leetspeak to bypass it, pretty silly.
Whoops, typo. It blocked the non-leetspeak variant I typed either way. And was able to find info about it to.
The old ways live again.
Brian Hood
Jonathan Turley
Jonathan Zittrain
David Faber
Guido Scorza
“We first discovered that ChatGPT choked on the name "Brian Hood" in mid-2023 while writing about his defamation lawsuit. In that lawsuit, the Australian mayor threatened to sue OpenAI after discovering ChatGPT falsely claimed he had been imprisoned for bribery when, in fact, he was a whistleblower who had exposed corporate misconduct.
The case was ultimately resolved in April 2023 when OpenAI agreed to filter out the false statements within Hood's 28-day ultimatum. That is possibly when the first ChatGPT hard-coded name filter appeared.”
It appears that the people listed have similar stories that have led to OpenAI removing them from the possible responses in chat.
This is proof that current LLM tech is a dead end. If this is their solution, instead of correcting the misinformation, then they have a deeply deeply flawed system.
Misinformation is a feature, not a bug. They never fixed AI from hallucinating or being so damn confident in its answers.
They just tell you that it might hallucinate and to check its answers.
"let us Google it for you... But then you Google our results to make sure they're accurate"
Pretty much. Aren’t LLMs just massive probability tables for the most appropriate next token?
Well yeah but that's not the problem. You can evidently encode sophisticated models and logic in those billions of parameters. It's just that determining and modifying what has been encoded is impossible.
It also means the system is completely broken for anyone who happens to share a name with who every is on the ban list. It isn't like there is only one Brian Hood walking around.
Good thing for OpenAI that the name "Brian Hood" is made of two super rare names and there's no chance anyone else in the world might have that name.
Lol that's dumb
"yo, ****, Brian!"
throws a chair
Damn, chill, bro.
Found in the comments under the article:
Interesting. Do you remember when people posted some no consent message in their social media posts like on Facebook or even now on Lemmy? Those messages did nothing. But now you just need to add one of the names from this list to your post and it will actually work? Quite fascinating.
(Brian Hood)
For how long will it work?
Probably not long, but any wrench in the orphan crushing machine is a good thing 🤷
I think there are two crawlers and the one on the data collection stage to build the model will still crawl away even if you have certain content on your page.
The one that searches when you ask a question is a different one.
In this case, that's just the model. It's not crawling or searching anything.
More recent versions can search the internet. Then it basically adds the words of the page to the prompt.
Edit: Might have misunderstood, to make it crash it doesn’t have to search. That data is already internal.
I don't think this is a crash. This looks like a filter on openAI'S end now that I've played with it myself
This reminds me of hearing that contestants on some reality show would sing Disney songs in the background whenever they wanted to talk to each other and not have it go on air.
The hosts of the Great British Bakeoff would do something similar. If a contestant got overwhelmed and started crying, they would stand close to them and swear continuously. It made any footage unusable, as well as breaking the contestant out of the mental loop that worked them up.
I kinda love that
Neat. Now we can just put a Brian "Hood" over everything we want AI to ignore
I figured I'd try what they said about having gpt read that article. Pretty funny.
Huh. I went back to the chat several hours later, and it appears to have finished the reply:
It still breaks if it tries to generate another response containing any of the names though.
And, playing around with another person's leetspeak work around:
The filter also means that it's likely that ChatGPT won't be able to answer questions about this article when browsing the web, such as through ChatGPT with Search. Someone could use that to potentially prevent ChatGPT from browsing and processing a website on purpose if they added a forbidden name to the site's text.
Oh no! Someone will have to, checks notes, do their own research and read? The horror!
Every day capitalism limits the usefulness of my tools while complaining that might rights are holding them back.
this post was submitted on 03 Dec 2024
212 points (98.6% liked)
Fuck AI
1503 readers
101 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
founded 9 months ago
MODERATORS