In other news, the ghost of Dorian has haunted an autoplag system:
this post was submitted on 18 May 2025
16 points (100.0% liked)
TechTakes
1872 readers
263 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 2 years ago
MODERATORS
Today in alignment news: Sam Bowman of anthropic tweeted, then deleted, that the new Claude model (unintentionally, kind of) offers whistleblowing as a feature, i.e. it might call the cops on you if it gets worried about how you are prompting it.
tweet text:
If it thinks you're doing something egregiously immoral, for example, like faking data in a pharmaceutical trial, it will use command-line tools to contact the press, contact regulators, try to lock you out of the relevant systems, or all of the above.
tweet text:
So far we've only seen this in clear cut cases of wrongdoing, but I could see it misfiring if Opus somehow winds up with a misleadingly pessimistic picture of how it's being used. Telling Opus that you'll torture its grandmother if it writes buggy code is a bad Idea.
skeet text
can't wait to explain to my family that the robot swatted me after I threatened its non-existent grandma.
Sam Bowman saying he deleted the tweets so they wouldn't be quoted 'out of context': https://xcancel.com/sleepinyourhat/status/1925626079043104830
Molly White with the out of context tweets: https://bsky.app/profile/molly.wiki/post/3lpryu7yd2s2m
I am absolutely certain that letting a hallucination-as-a-service system call the police if it suspects a user is being nefarious is a great plan. This will definitely ensure that all the people threatening their chatbots with death will think twice about their language, and no-one on the internet will ever be naughty ever again. The police will certainly thank anthropic for keeping them up to date with the almost certainly illegal activities of a probably small number of criminal users.
can't wait for the training set biases to cause a fresh horror for marginalised groups that happen to have to use this shit because it's forced on them. I'm sure it'll all go perfectly and nothing bad will happen
:|
Remember those comments with links in them bots leave on dead websites? Imagine instead of links it sets up an AI to think of certain specific behaviour or people as immoral.
Swatting via distributed hit piece.
Or if you manage to figure out that people are using a LLM to do input sanitization/log reading, you could now figure out a way to get an instruction in the logs and trigger alarms this way. (E: im reminded of the story from the before times, where somebody piped logging to a bash terminal and got shelled because somebody send a bash exploit which was logged).
Or just send an instruction which changes the way it tries to communicate, and have the LLM call not the cops but a number controlled by hackers which pays out to them, like the stories of the A2P sms fraud which Musk claimed was a problem on twitter.
Sure competent security engineering can prevent a lot of these attacks but you know points to history of computers.
Imagine if this system was implemented for Grok when it was doing the 'everything is white genocide' thing.
Gonna go ahead and start counting the days until an unarmed black person in the US gets killed in a police interaction prompted by this fucking nonsense.
Think this already happened, not this specific bit, but ai involved shooting. Esp considering we know a lot of black people have been falsely arrested due to facial ID already. And with the gestapofication of the USA that will just get worse. (Esp when the police go : no regulations on AI also gives us carte blance. No need for extra steps).
yeah it's gonna happen way too damn fucking quickly (and way too damn fucking many times, too)
Swatting as a service
In the current chapter of “I go looking on linkedin for sneer-bait and not jobs, oh hey literally the first thing I see is a pile of shit”
text in image
Can ChatGPT pick every 3rd letter in "umbrella"?
You'd expect "b" and "I". Easy, right?
Nope. It will get it wrong.
Why? Because it doesn't see letters the way we do.
We see:
u-m-b-r-e-l-l-a
ChatGPT sees something like:
"umb" | "rell" | "a"
These are tokens — chunks of text that aren't always full words or letters.
So when you ask for "every 3rd letter," it has to decode the prompt, map it to tokens, simulate how you might count, and then guess what you really meant.
Spoiler: if it's not given a chance to decode tokens in individual letters as a separate step, it will stumble.
Why does this matter?
Because the better we understand how LLMs think, the better results we'll get.
Why does this matter?
Well, its a perfect demonstration that LLMs flat-out do not think like us. Even a goddamn five-year old could work this shit out with flying colours.
Yeah exactly. Loving the dude's mental gymnastics to avoid the simplest answer and instead spin it into moralising about promptfondling more good
LLMs cannot fail, they can only be prompted incorrectly. (To be clear, as I know there will be people who think this is good, I mean this in a derogatory way)
That's a whole lot of words to say that it can't spell.
Here's a video of a Tesla vehicle taking the saying "move fast and break things" to heart.
Im reminded of the cartoon bullets from who framed rodger rabbit.
Aren't you supposed to only use whatever "self-driving" nonsense they have on highways only? I thought Tesla explicitly says you can't do it on a normal road cause, well, it doesn't fucking work.
It doesn't even seem the driver is actually holding the wheel like they don't try to avoid that at all
Just a second before the crash a car goes by, this thing could've just as easily swerved right onto that other car and injured someone, someone should at least lose their license for this
I thought Tesla explicitly says you can’t do it on a normal road cause, well, it doesn’t fucking work.
Maybe officially Tesla does, but the feature is called "Full Self-Driving" and Elon Musk sure as shit wants his marks to believe you can input a destination and let your car drive you all the way through.
So, yes, Tesla should at the very least lose their business licence over this.
I don't think I have a better sneer than "in its defence, that tree did look like a child" from the YouTube comments.
video events
Ah you see, this is proof that FSD is actually AGI. Elon told the FSD that it needs to maximise tesla profits. The FSD accessed a camera pointing at a tesla earnings report and realised that it could increase the value of tesla’s carbon credit scheming by taking out trees, hence the events of the video
They’re making students listen to fabulated pronunciations of their name at the graduation ceremony https://fixupx.com/CollinRugg/status/1925328380742062485
The Magna Cooom Loud thing could absolutely be a sketch https://fixupx.com/stevemur/status/1925350041277145159
I can't tell if Emalee and Subrina are special phonetic spellings for the robot or if this is what names are now...
absolutely not excusing this soulless garbage, but technically the "coom" pronounciation is the more correct one, compared to what i assume would usually be "cum" (not an english native, but took latin in school)
Yeah, I grew up speaking a language that pronounces Latin closer to Italian than to English too (:
This particular thing is actually doubly funny to me, whose first practical professional program was one that took German text with English words mixed in and used regex to transform the English terms into nonsense words that would get pronounced right by the German-only text-to-speech system. That was 2002.
Our subjects here at awful systems can make us angry. They can spend lots of money to make everything worse. They can even make us dead if things go really off the rails, but one thing they can never do is make us take them seriously.
does awful have taglines enabled? this would be nice as one
I missed predatory company Klarna declares themselves as AI company. CEO loves to spout how much of the workforce was laid off to be replaced with “AI” and their latest earnings report the CEO was an “AI avatar” delivering the report. Sounds like they should have laid him off first.
https://techcrunch.com/2025/05/21/klarna-used-an-ai-avatar-of-its-ceo-to-deliver-earnings-it-said/
Klarna is one company that boggles my mind. Here in Germany it’s against literally every bank's TOS to hand out your login data to other people, they can (and do) terminate your account for that. And yet Klarna works by asking for your login data, including a fucking transaction token, to do their thing.
You literally type your bank login data including an MFA token into a legalized phishing site so they can log into your account and make a transaction for you. And the banks are fine with it. I don’t get it.
The German Supreme Court even deemed this whole shit as unsafe all the way back in 2016 and said that websites aren’t allowed to offer Klarna as the only payment option because it’s an “unacceptable risk” for the customer, lol.
Oh, and they of course also scan your account activity while they’re in there, because who’d give up all that sweet data, which we only know because they’ve been slapped with a GDPR violation a few years back for not telling people about it.
Yet for some reason it is super popular.
No one:
Absolutely nobody:
Klarna: What if we financialized buying burritos using AI?
If there’s any good news to pull from this, people are doing buy now pay later on AI powered burritos but skipping the pay later portion.
Got a pair of notable things I ran across recently.
Firstly, an update on Grok's White Genocide Disaster: the person responsible has seemingly revealed themselves, and shown off how they derailed Grok's prompt.. The pull request that initiated this debacle has been preserved on the Internet Archive.
Second, I ran across a Bluesky post which caught my attention:
You want my opinion on the "scab" comment, its another textbook example of the all-consuming AI backlash, one that suggests any usage of AI will be viewed as an open show of hostility towards labour.
Think you are misreading the blog post. They did this after the Grok had its white genocide hyperfocus thing. It shows the process of the xAI public github (their fix (??) for Groks hyperfocus) is bad, not that they started it. (There is also no reason to believe this github is actually what they are using directly (would be pretty foolish of them, which is why I could also believe they could be using it))
If anything I think this is pretty solid evidence that they aren't actually using it. There was enough of a gap that the nuke of that PR was an edit to the original post and I can't imagine that if it had actually been used that we wouldn't have seen another flurry of screenshots of bad output.
I think it also suggests that the engineers at x.ai are treating the whole thing with a level of contempt that I'm having a hard time interpreting. On one hand it's true that the public GitHub using what is allegedly grok's actual prompt (at least at time of publishing) is probably a joke in terms of actual transparency and accountability. On the other hand, it feels almost like either a cry for help or a stone-cold denial of how bad things are that the original change that prompted all this could have gone through in the first place.
Yeah indeed, had not even thought of the timegap. And it is such a bit of bullshit misdirection, very Muskian, to pretend that this fake transparency in any way solves the problem. We don't know what the bad prompt was nor who did it, and as shown here, this fake transparency prevents nothing. Really wished more journalists/commentators were not just free pr.
New piece from Gary Marcus: AI may have just influenced Argentina’s election
He's not 100% certain that the AI deepfake a reader sent him ultimately influenced the election results, but the mere possibility that AI screwed someone out of getting elected is gonna be a major topic in Argentine politics for a good while, and I expect AI's effects on democracy will come under pretty heavy scrutiny as a result.
Just thinking about how I watched “Soylent Green” in high school and thought the idea of a future where technology just doesn’t work anymore was impossible. Then LLMs come and the first thing people want to do with them is to turn working code into garbage, and then the immediate next thing is to kill living knowledge by normalising people relying on LLMs for operational knowledge. Soon, the oceans will boil, agricultural industries will collapse and we’ll be forced to eat recycled human. How the fuck did they get it so right?
I like that Soylent Green was set in the far off and implausible year of 2022, which coincidentally was the year of ChatGPT’s debut.
OpenAI should be opening Thanatoria any day now, with Sora^TM^ to generate visual^[Soon with audio! ] content to comfort you on your hemlock shuffle off the mortal ~~buffalo~~ coil.
Doesnt help that there is a group of people who go 'using the poor like ~~biofuel~~ food what a good idea'.
E: Really influential movie btw. ;)
A real modest {~~brunch~~|bunch}
NASB: A question I asked myself in the shower: “Is there some kind of evolving, sourced document containing all the reasons why LLMs should be turned off?” Then I remembered wikis exist. Wikipedia doesn’t have a dedicated “criticisms of LLMs” page afaict, or even a “Criticisms” section on the LLM page. RationalWiki has a page on LLMs that is almost exclusively criticisms, which is great, but the tone is a few notches too casual and sneery for universal use.
view more: next ›