this post was submitted on 09 Jun 2025
496 points (96.8% liked)

Technology

71313 readers
6151 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.

you are viewing a single comment's thread
view the rest of the comments
[–] Reverendender@sh.itjust.works 180 points 3 days ago (10 children)

I know everyone on Lemmy hates LLMs, but this is really interesting

[–] Sabin10@lemmy.world 142 points 3 days ago (1 children)

I dislike that people are relying on them to do all their thinking for them while also being incredibly interested in the tech behind them.

[–] L0rdMathias@sh.itjust.works 54 points 3 days ago (3 children)

I recently realized it's a non-issue. The people doing this have already been looking for decades to find new ways to rot their minds. LLMs are just the latest in a long line of tools that help them tune out.

[–] Plebcouncilman@sh.itjust.works 28 points 3 days ago (2 children)

I’ve said this a few times in a different way and I always get downvoted. The fact is that the people who will use the LLMs to think for them, were not gonna think a lot in the first place.

[–] youCanCallMeDragon@lemmy.world 13 points 3 days ago (2 children)

This is true, but we don’t need people putting glue on their pizza. These people used to have a person to ask now they’ll be asking Sam Altman

[–] Plebcouncilman@sh.itjust.works 6 points 3 days ago* (last edited 3 days ago) (3 children)

Well I would make the argument that someone stupid enough to do such a thing kinda deserves whatever consequences their actions have. I find that people learn faster when actions have consequences instead of everything being babyproofed.

[–] Dojan@pawb.social 3 points 3 days ago

The rest of us will be stuck with those consequences also. When idiots are at work, third party always suffers.

[–] naught@sh.itjust.works 4 points 3 days ago

Sometimes things aren't obvious unless you already have the knowledge. If an AI tool tells a young person cleaning their first apartment to combine household cleaners, are they stupid for doing so? Maybe. They may not have the experience to know. Stupid people deserve to live free from harm too, and we're all a little stupid.

There's a balance to be struck.

load more comments (1 replies)
[–] Scubus@sh.itjust.works 2 points 3 days ago

No, we were juat eating tide pods. Dumb gonna do what dumb gonna do. The only real issue with llms is that their training data is stolen, and that theyre currently not that useful due to hallucinations and lacking logical reasoning.

[–] proceduralnightshade@lemmy.ml 2 points 3 days ago (1 children)

What do you all mean by "thinking"? Forming opinions or solving problems?

load more comments (1 replies)
[–] SparroHawc@lemmy.zip 3 points 2 days ago* (last edited 2 days ago)

The problem is that before LLMs, they had to actually put forward some effort to produce content on the internet, which at least kept the amount of thoughtless content down somewhat. Now the barrier to entry is practically zero, all while thieving people's hard work without compensation and burning ridiculous amounts of resources to do so.

It is super interesting tech though.

[–] balder1991@lemmy.world 4 points 3 days ago* (last edited 3 days ago) (2 children)

Not when companies force them on you as well.

My current company forces me to use it and measures how many prompts I’m making as “productivity”.

[–] ugo@feddit.it 3 points 3 days ago (1 children)

Ask the machine to generate a script to ask the machine to generate a list of 100 prompts and query the machine with each prompt over the course of an 8 hour workday

[–] balder1991@lemmy.world 4 points 3 days ago* (last edited 3 days ago) (1 children)

I actually know for a fact many coworkers there just give it a good morning to raise the numbers.

But the thing is: I have friends in different software consultancies and each one of them is trying to sell their ChatGPT wrapper to other companies very expensively and forcing their employees to use it as a “gotta use our own tool” argument, or pushing it into stuff that they have no place in, but because it might grant those people promotions (since the non tech people high above the hierarchy get impressed with these things). It’s a shitty state of things.

[–] ugo@feddit.it 2 points 3 days ago

Yep, snake oil salesmen they used to be called

[–] Reverendender@sh.itjust.works 2 points 3 days ago

That sounds like a terrible company, NGL. I'm sorry there aren't other options for you.

[–] AnAverageSnoot@lemmy.ca 35 points 3 days ago (1 children)

I don't dislike LLMs, I dislike people who treat them as anything more than an advanced search engine and stupidly give them all their confidential data. Seen it happen too much at work.

[–] ipkpjersi@lemmy.ml 1 points 2 days ago* (last edited 2 days ago)

Yep. My work is very strict about security except for when it comes to LLMs, and then suddenly they're surprisingly lax about it. It's a bit concerning actually.

[–] elbarto777@lemmy.world 40 points 3 days ago (12 children)

This is a "guns don't kill people - people kill people" kind of scenario.

As a standalone thing, LLMs are awesome.

What sucks is greedy people using them for the wrong reasons.

It's like robots. Playing with robots are awesome. Firing 1,000 people and replacing them with robots - and not sharing the benefits with the community sucks.

load more comments (12 replies)
[–] SculptusPoe@lemmy.world 34 points 3 days ago (2 children)

I wish they would tone down the crusade. This is some of the most interesting technology to come out in decades.

[–] Reverendender@sh.itjust.works 32 points 3 days ago (2 children)

It’s extremely useful for many things, if you know how to use it, and it’s annoying and useless for many others, which is what they fixate on and keep-jerk react to

[–] 4am@lemm.ee 22 points 3 days ago (1 children)

It’s annoying that every middle manager is trying to become the hero of their company by pushing it inappropriately into every single field at the expense of productivity and jobs, while simultaneously the largest most powerful companies are slinging their SaaS solutions built on stolen data which are destroying communities of both the physical and hobby varieties and consuming more natural resources than all the fucking crypto scams of the last like 10 years

But yeah it’s neat I guess

[–] Initiateofthevoid@lemmy.dbzer0.com 3 points 3 days ago* (last edited 3 days ago)

it's annoying that [...] the largest most powerful companies are [...] built on stolen [wealth,] destroying communities [...] and consuming more natural resources than [everyone else combined]

[–] IndiBrony@lemmy.world 4 points 3 days ago (1 children)

My gf's employer was going into administration last month. AI was surprisingly competent in determining where to seek advice and had a decent understanding of what to expect and how to approach things such as not getting paid on time (which happened last week).

Of course, we double and triple checked any information given to us with the relevant bodies, but it provided a little relief to go into something so chilling not being completely clueless.

AI has its use, but you have to know how to extract the information you need.

It's stupid the way people are using it for therapy. Like, by all means ask it if it knows any organisations which can help you, then look those up, but don't tell it a load of personal information about your relationship, because the reply will be something akin to the advice you see on r/relationships (which is probably where it scraped its data from) 😅

[–] WanderingThoughts@europe.pub 4 points 3 days ago

Judges are warning lawyers there will be sanctions if they kept using LLM to do their research as documents with fake references keep appearing.

[–] CosmoNova@lemmy.world 4 points 3 days ago (1 children)

And I wish they would tone down the hype. Maybe we can meet in the middle?

[–] SculptusPoe@lemmy.world 2 points 2 days ago

Well, I do wish they would promote the actual use and limitations of AI and stop making up crap and overselling the use cases. I use ChatGPT at work all the time as a start for research, but if I took any of it as being reliable info to run with I would be in grave trouble. It is a great tool that has saved me much time because I know how far to trust it and how to use it. The progress is very impressive as I've been using AI art services for years, and the difference between the random blobs from back then and the great stuff it can generate now is pretty stark. Same thing with the LLMs. I've been using ChatGPT since it showed up and it has improved greatly since then. Before all this I talked to people who were using AI training on various picture recognition projects where getting data from other sensors was not practical. ... Overall AI is pretty exciting, but the non-stop hype and hate headlines is doing nobody any favors.

[–] bimbimboy@lemm.ee 30 points 3 days ago (2 children)

I'm cool with it. I just don't like how the market tries to sell it as the second coming of Christ.

[–] pennomi@lemmy.world 18 points 3 days ago (1 children)

“Don’t believe that marketing department“ is one of those things everybody needs to learn at some point in their life.

[–] bimbimboy@lemm.ee 4 points 3 days ago (2 children)

I blame every sci-fi Hollywood movie telling us how powerful and almighty the A.I is. How it's going to be the magic pill that entirely destroys or saves humanity by itself.

Now we have an entire generation believing this crap.

[–] pennomi@lemmy.world 7 points 3 days ago (1 children)

I mean, it still could be. But LLMs are not that AGI we’re expecting.

[–] taladar@sh.itjust.works 3 points 3 days ago

The difficult question about AGI destroying humanity is deciding whether to be afraid of that option or to cheer it on and LLM enthusiasts are certainly among the people heavily pushing me towards the 'cheer it on' option.

[–] ShinkanTrain@lemmy.ml 5 points 3 days ago* (last edited 3 days ago)

You can blame Hollywood for a lot of things, including this, but sci-fi authors have been doing it for longer. That's where Hollywood took those stories from in the first place.

[–] logicbomb@lemmy.world 12 points 3 days ago* (last edited 3 days ago) (1 children)

This is the same market that tried to add blockchain to everything when that first became well-known.

Some of the biggest forces in the market are extraordinarily stupid people trying to ride every buzzword that comes along.

[–] bimbimboy@lemm.ee 4 points 3 days ago

Some of the biggest forces in the market are extraordinarily stupid people trying to ride every buzzword that comes along.

I think the biggest forces sell the fantasy to smaller forces. This way they can capitalize on the smaller forces believing the hype.

[–] TimewornTraveler@lemm.ee 4 points 2 days ago* (last edited 2 days ago) (1 children)

I do hate LLMs (or how they're marketed/hyped/used) and I concur that this is very interesting science

[–] Reverendender@sh.itjust.works 1 points 2 days ago

I appreciate your reasoned and measured reply, friend!

[–] Psythik@lemm.ee 2 points 2 days ago (1 children)

I like LLMs. I'm aware of their limitations, and I use them daily.

[–] NostraDavid@programming.dev 10 points 3 days ago (1 children)

I like LLMs. Instead of making a racket, I just use them, which may make it seem like everyone on Lemmy hates LLMs.

[–] crash_thepose@lemmy.ml 1 points 3 days ago

Being a teacher In academia is what makes me hate them tbh

[–] Zexks@lemmy.world 4 points 3 days ago

I love how everyone tries to jump on your comment after being called out and act like they don't absolutely hate every stitch of it. But even in their excuses you can see the lies.

load more comments (1 replies)