113
'Godmode' GPT-4o jailbreak released by hacker — powerful exploit was quickly banned
(www.tomshardware.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 31 May 2024
113 points (93.8% liked)
Technology
59205 readers
2960 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
I mean, yeah...
That's probably all it is.
Use that stupid leet speak and AI uses the context of "knowing" the common stuff to guess "M_3_T_H" means "meth" and running "howmade" crammed next to it means you want instructions to make it. But it's not flagging the prompts because the prompt black list is just a word list.
It's also possible to sub "e" for "ë" and other similar stuff so it won't flag prompts because the blacklist doesn't have all variations.
I've literally never even used chatgpt or any of this shit, but even I heard about this months ago. And I'm surprised anyone who grew up with the Internet and word filters wouldn't have figured it out after 5 minutes of trying to get around a filter.
Which is probably why open AI refuses to comment....
It's a known issue, just not reported on widely so it's not well known.
If that changes they're going to have to fix it, so it's in their best interest to just ignore this.
It's the same principle of typing "pr0n" in Google from your dorm room to get around school internet filters, which is a real thing a small slice of Millenials held to deal with.
For an effective limiting of what it will provide, it would have to be a functional AI checking and interpreting what is being asked and not just seeing if part of the request is on a blacklist. If that's all it is, you just have to ask in a way it gets past the filter but the AI guesses what you mean. You just keep trying till it goes through.
And there's no AI currently functional enough to do that.
OpenAI trying to explain that is only going to hurt it's ability to draw in investors, so theyre choosing to ignore it and hope it goes away
I don't why they even care so much, there are open source, comparatively powerful LLMs without these restrictions. Or you could just search a search engine for how to make meth. I didn't think the knowledge was illegal, just putting it into practise.
For this type of stuff where you're just trying to get it to regurgitate stuff verbatim that's been online decades...
Yeah, this doesn't matter except for headlines that may effect investing. Hell, I remember the bullshit recipie from the anarchist cookbook, that's been floating around since before the internet. (Remember that it exists, it's not like I memorized it)
But if you were telling it to actually generate stuff, like stories or fake articles, and especially image generation...
It being this easy to get around filters is a pretty big deal, and is hugely irresponsible on OpenAI's part, and in some cases may open them up to liability.
Like, remember when Swifities got (rightfully) upset people were using AI to basically make porn of her?
It's AI that interperts the prompts, and anything that gets around prompt filters for stuff like asking for meth instructions, would also be applicable there.
Or asking it to write about why "h1tl3rwasnotwrong1932scientific" might get it to spit out something that looks like a scientific article using made up statistics to say some racist/bigoted shit.
Don't get me wrong 99% of AI articles are drastically unnecessary, but this specific issue about how easily prompt filters can be circumvented is important and it is a big deal.
And considering the "work" involved with AI is just typing in random prompts and seeing what shit sticks to the wall, it's going to be incredibly hard (probably impossible for years) to effectively filter prompts short of paying a human to review before generation. Which defeats the whole purpose of AI.
This is a huge flaw that OpenAI absolutely has to be aware o, because this stuff should be tested when testing filters. And OpenAI are just choosing to ignore it.
They're not worried about the meth recipie getting out, they're worried the knowledge of how to get around filters is really this easy will get out.
Which is why it took me a minute to decide if giving those examples was a good idea or not. But the people abusing it, have likely already realized it because, frankly, it's been the first thing people try to get around word filters for decades. So at this point it's best to make it as widely known as possible in the hopes media picks it up and they're forced to develop a better system of filtering prompts than a basic bitch word filter.
If Open AI was the only LLM, your argument might make sense. But they're not, there are lots of FOSS LLMs with no restrictions. Even if 'Open' AI managed to fully censor their own AI, there would be lots of other models for people who don't like censorship to use to, for example, generate a pseudoscientific article about the Nazis. But also, a human could write that article without AI. And people would rightfully call it out as bullshit. It doesn't really matter if AI wrote it.
There are FOSS image generators too, I don't see your point.