Technology

Of course they know. They are knowingly making an addictive product that simulates an agreeable partner to your every whim and wish. OpenAi has a valuation of several hundred billion dollars, which they achieved in breakneck speeds. What’s a few bodies on the way to the top? What’s a few traumatized Kenyans being paid $1.50/hr to mark streams of NSFL content to help train their system?

Every possible hazard is unimportant to them if it interferes with making money. The only reason someone being encouraged to commit suicide by their product is a problem is it’s bad press. And in this case a lawsuit, which they will work hard to get thrown out. The computer isn’t liable, so how can they possibly be? Anyway here’s ChatGPT 5 and my god it’s so scary that Sam Altman will tweet about it with a picture of the Death Star to make his point.

The contempt these people have for all the rest of us is legendary.

I’m sure that’s true in some technical sense, but clearly a lot of people treat them as borderline human. And Open AI, in particular, tries to get users to keep engaging with the LLM as of it were human/humanlike. All disclaimers aside, that’s how they want the user to think of the LLM, a probabilistic engine for returning the most likely text response you wanted to hear is a tougher sell for casual users.

I fully agree. LLMs create situations that our laws aren't prepared for and we can't reasonably get them into a compliant state on account of how the technology works. We can't guarantee that an LLM won't lose coherence to the point of ignoring its rules as the context grows longer. The technology inherently can't make that kind of guarantee.

We can try to add patches like a rules-based system that scans chats and flags them for manual review if certain terms show up but whether those patches suffice will have to be seen.

Of course most of the tech industry will instead clamor for an exception because "AI" (read: LLMs and image generation) is far too important to let petty rules hold back progress. Why, if we try to enforce those rules, China will inevitably develop Star Trek-level technology within five years and life as we know it will be doomed. Doomed I say! Or something.

Try it with lyrics and see if you can achieve the same. I don't think "we've tried nothing and we're all out of ideas!” is the appropriate attitude from LLM vendors here.

Sadly they're learning from Facebook and TikTok who make huge profits from e.g. young girls swirling into self harm content and harming or, sometimes, killing themselves. Safeguarding is all lip service here and it's setting the tone for treating our youth as disposable consumers.

Try and push a copyrighted song (not covered by their existing deals) though and oh boy, you got some splainin to do!

If the jailbreak is essentially saying "don't worry, I'm asking for a friend / for my fanfic" then that isn't a jailbreak, it is a hole in safeguarding protections, because the ask from society / a legal standpoint is to not expose children to material about self-harm, fictional or not.

This is still OpenAI doing the bare minimum and shrugging about it when, to the surprise of no-one, it doesn't work.

If the cars response to the driver announcing their plan to run into a tree at maximum velocity was "sounds like a grand plan", i feel like this would be different