171

Court ordered penalties for 15 teens who created naked AI images of classmates (arstechnica.com)

submitted 5 months ago by sabreW4K3@lazysoci.al to c/technology@beehaw.org

55 comments fedilink hide all child comments

top 50 comments

sorted by: hot top controversial new old

[-] Ilandar@aussie.zone 67 points 5 months ago

I'm surprised by some of the comments here. So much hatred for the teenagers but very little reflection on the responsibility of us as adults to create a world which is safe and understandable for young people. The real story here is not about the actions of the boys, it is about the actions of the people operating the websites they used, the big tech companies who created the tools and the governments (which we adults vote for) failing to properly regulate any of this. Generative AI is a disaster for young people and it was adults who created and unleashed it on them.

[-] Michal@programming.dev 21 points 5 months ago

We (adults) created a world with knives, guns, automobiles. But if any of these were used by a child for manslaughter would you still blame collective adults? No, parents are held responsible for protecting their children and controlling their access to dangerous tools.

[-] Ilandar@aussie.zone 14 points 5 months ago* (last edited 5 months ago)

Bad argument. None of those things are easily accessible by, or even targeted at, teenagers. Generative AI is a technology that big tech is specifically building into the social media platforms and devices that young people use every day and there is no regulation or education to protect against the dangers it poses.

load more comments (1 replies)

[-] mrpants@midwest.social 11 points 5 months ago

Yeah. Society is responsible for the outcomes of its children. It shouldn't personally hurt your feelings but it should motivate your actions.

[-] tesseract@beehaw.org 6 points 5 months ago

I'm an adult and am not responsible for anything you described. They were all there even before I was born. In fact, the same may apply to my parents or even grandparents. I'd rather blame a sociopolitical class than any single generation for all those ills.

But to answer your question, yes, I'd blame that entire class for the harm caused by young people using murder tools they introduced. They did it with the full knowledge of its consequences. They valued momentary material gains above the wellbeing of entire generations. They absolutely should be punished for all the mass shootings in schools, because they knew it could happen. Yet they chose the blood money. Similarly, if an entire city is under a drugs epidemic (like the current opioid crisis), wouldn't you want to hunt down the producers and suppliers, instead of the users?

[-] Umbrias@beehaw.org 3 points 5 months ago

I mean I would and do in fact literally blame societal and familial problems when kids are brutal, unkind, or hurt others, and similarly blame societal and familial problems for when kids are not protected from brutal, unkind, and hurtful things.

Why are you saying the things you're saying like a gotcha? Do you not feel that society has a significant impact on the behavior of youth?

[-] awesome_lowlander@lemmy.dbzer0.com 14 points 5 months ago

Would the impact have been any different if they had used photoshop?

[-] Ilandar@aussie.zone 7 points 5 months ago

Since when could someone do this in Photoshop with a couple of clicks and zero training, for free and on any device? Since when has Adobe specifically marketed Photoshop as a tool for creating sexually explicit images?

[-] awesome_lowlander@lemmy.dbzer0.com 14 points 5 months ago

Since when could someone do this ... with a couple of clicks and zero training, for free and on any device?

You know, somebody probably said the exact same thing about Photoshop when it first came out? Back when cutting up photos and pasting them together was a thing. Then again, somebody probably said the same thing about the advent of photography too, during the days of woodcuts and oil paintings.

[-] Ilandar@aussie.zone 3 points 5 months ago

Since when has Adobe specifically marketed Photoshop as a tool for creating sexually explicit images?

[-] awesome_lowlander@lemmy.dbzer0.com 20 points 5 months ago

Fuck knows. It's not like I made any such claim, you just decided to put up a strawman argument.

[-] Ilandar@aussie.zone 8 points 5 months ago

How is it a "strawman" when it relates specifically to the topic being discussed? A simple web search can bring up countless examples of generative AI tools that are designed to "undress" or "nudify" women. You seem to believe there is zero difference between using one of these sites and using Photoshop, so I am asking when Adobe marketed Photoshop as an undressing tool. The ease of use and access is the key difference here that you are trying to avoid acknowledging.

[-] awesome_lowlander@lemmy.dbzer0.com 15 points 5 months ago

My question was very simple and limited in scope:

Would the impact have been any different if they had used photoshop?

I did not make any sweeping comments, and I would appreciate if you stopped trying to claim things on my behalf.

[-] Ilandar@aussie.zone 3 points 5 months ago

They didn't use Photoshop, they used generative AI. Maybe you should spend a little less time trying to save face and a little more time thinking about why that might have been.

[-] awesome_lowlander@lemmy.dbzer0.com 9 points 5 months ago

Right. I see there's no logical discussion to be had here. Have a good day.

load more comments (2 replies)

load more comments (3 replies)

[-] thegr8goldfish@startrek.website 39 points 5 months ago

I don't understand why using AI is what makes this illegal. I don't know the laws in Spain, but would it be illegal if they used a pencil or a paint brush? Seems like a weird line to draw if not.

[-] Ava@beehaw.org 48 points 5 months ago* (last edited 5 months ago)

The minors were charged with 20 counts of creating child sex abuse images and 20 counts of offenses against their victims’ moral integrity.

The article doesn't make the claim that the AI is what makes it illegal, simply that AI was used. It's literally the second sentence. Indeed, it goes on to highlight that there are legal novelties prosecuting the use of AI.

[-] Darkassassin07@lemmy.ca 30 points 5 months ago

As far as I understand; it's not the tools used that makes this illegal, but the realism/accuracy of the final product regardless of how it was produced.

If you were to have a high proficiency with manual Photoshop and produced similar quality fakes, you'd be committing the same crime(s)

creating child sex abuse images

and

offenses against their victims’ moral integrity

The thing is, AI tools are becoming more and more accessible to teens. Time, effort, and skill are no longer roadblocks to creating these images; which leaves very very little in an irresponsible teenagers way...

[-] CanadaPlus@lemmy.sdf.org 5 points 5 months ago* (last edited 5 months ago)

Which still seems kinda dumb. How realistic is too realistic? You could make a legal standard of "photography-like", or something, just to define who to convict, but you still haven't really justified why.

The sentence in this case is just classes, though, so I'll leave my pitchfork in the shed.

[-] Darkassassin07@lemmy.ca 20 points 5 months ago

Did... Did you just ask; why creating photo-realistic sexually explicit material of real children, should be illegal?

[-] CanadaPlus@lemmy.sdf.org 12 points 5 months ago* (last edited 5 months ago)

Keep in mind these were other kids their age. We're not talking about pedo stuff here.

All the recent stuff about deepfakes feels a bit moral-panic-y to me. I think we should have a better reason than just ick before anyone gets thrown in jail.

[-] Kissaki@beehaw.org 17 points 5 months ago

We’re not talking about pedo stuff here.

Do you want an explanation of why creating and sharing sexually explicit material of other people without consent is problematic and damaging, and especially for children?

[-] Eggyhead@kbin.run 5 points 5 months ago

This is a really good idea. Perhaps this is what should be happening in the first place rather than resorting to direct legal enforcement, which can be problematic and damaging, especially for children.

[-] Zoot@reddthat.com 9 points 5 months ago

If you cant understand that sharing naked photos of people is bad, then you probably should have to face the court systems.

Like what? I don't care how horny you are as a teenager, it takes a real fucking idiot, and a huge shitstain to go and share those photos. They absolutely deserve the book being thrown at them.

load more comments (2 replies)

load more comments (1 replies)

[-] DragonTypeWyvern@midwest.social 35 points 5 months ago

Not the company making the CSAM machine though.

[-] mozz@mbin.grits.dev 34 points 5 months ago

In addition to probation, the teens will also be required to attend classes on gender and equality, as well as on the "responsible use of information and communication technologies,"

…

What?

Have you not interacted with teenage boys?

I can think of not much more of a better way to teach them there are no consequences and they can keep doing this as long as they smirk and say they’re sorry whenever they get caught

[-] OsrsNeedsF2P@lemmy.ml 47 points 5 months ago

The minors were charged with 20 counts of creating child sex abuse images and 20 counts of offenses against their victims’ moral integrity

Punishment or not, those charges are still scary. I think the probation and courses are a good addition.

[-] mozz@mbin.grits.dev 10 points 5 months ago

I don’t think those are additions, I think those are the punishments for those charges, in full. I could be wrong but that’s how I read it.

[-] Chozo@fedia.io 31 points 5 months ago

Teens sentenced in Spain were between the ages of 13 and 15. According to the Guardian, Spanish law prevented sentencing of minors under 14, but the youth court "can force them to take part in rehabilitation courses."

Some of them are too young to receive real sentencing. It's important to remember that they're children, too.

[-] jonne@infosec.pub 19 points 5 months ago

Yeah, it's probably more important to make sure we don't have child porn generation machines available to anyone online.

[-] zurohki@aussie.zone 14 points 5 months ago

Since anyone can download and train their own AI, that ship has probably sailed.

[-] jonne@infosec.pub 11 points 5 months ago

I think training your own image generator on existing child porn is probably beyond most high schoolers. I'd be happy if at least commercial options were held responsible for distributing generated CP, which is already illegal BTW.

[-] cygnus@lemmy.ca 14 points 5 months ago

I don't think the models are trained on CP. They're likely trained on widely-available porn.

[-] zurohki@aussie.zone 9 points 5 months ago

This. If you ask an image generator for a bed in the shape of a pineapple, it probably has no pineapple-shaped beds in its training data but it has pineapples and beds and can mash the concepts together.

load more comments (2 replies)

[-] mozz@mbin.grits.dev 3 points 5 months ago* (last edited 5 months ago)

Good luck with that

I mean you can do a significant amount by making it illegal to offer it on the open web, which might be the way to go, but creating awesome things that can be had once you go outside the law actually carries its own little long-term consequences

[-] unconfirmedsourcesDOTgov@lemmy.sdf.org 28 points 5 months ago

I disagree, these children are minors and the their behavior, while abhorrent, belies a fundamental lack of perspective and empathy.

I've been a teenage boy before and I did some bone-headed things. Maybe not this bad, but still, I agree with the judge in this instance that it would be inappropriate to impose permanent consequences on these kids before their life even gets started because they were stupid, horny, teenage boys.

Even if we assume that these kids don't all have well-meaning parents who who will impose their own punishments, having a probation officer in high school is not going to help with popularity. Then, mandatory classes that will force these boys to evaluate the situation from another perspective seems like a great add-on.

I know it doesn't feel like justice, but our goal as a society shouldn't be to dole out maximum punishment in every instance. The goal is to allow all of us to peacefully coexist and contribute to society - throwing children in a dark hole somewhere to be forgotten isn't going to help with that.

Having said all of the above, it feels like a good time to emphasize that we still don't have any good ideas for solving the core problem here, which is the malicious use of this technology that was dumped on society without any regard for the types of problems that it would create, and entirely without a plan to add guard rails. While I'm far from the only one considering this problem, it should be clear enough by now that dragging our feet on creating regulation isn't getting us any closer to a solution.

At a minimum it feels like we need to implement a mandatory class on the responsible use of technology, but the obvious question there is how to keep the material relevant. Maybe it's something that tech companies could be mandated to provide to all users under 18 - a brief, recurring training (could be a video, idc) and assessment that minors would have to complete quarterly to demonstrate that they understand their responsibilities.

[-] mozz@mbin.grits.dev 18 points 5 months ago* (last edited 5 months ago)

I've been a teenage boy before and I did some bone-headed things. Maybe not this bad, but still, I agree with the judge in this instance that it would be inappropriate to impose permanent consequences on these kids before their life even gets started because they were stupid, horny, teenage boys.

Completely agree with 100% of this

I’m just saying that I think the answer lies somewhere between “take some classes and promise not to do it again” and “adult prison”. They imposed significant harm to another human being, in a way that’s so significant that we all agreed it should be illegal. Yes, I know that probably wasn’t the intent on their part. But this kind of “oh but I just got horny and just kind of didn’t care / wasn’t focused on what the impact was” is not a thing you wanna teach them there’s some wiggle room with as long as they make sure to apologize about it after.

Community service? Home arrest? Juvenile detention for 21 days? Fuckin something? I’m not saying put them in the hole.

[-] kent_eh@lemmy.ca 7 points 5 months ago

I've been a teenage boy before and I did some bone-headed things

Same.

I would be surprised if anyone with the same history didnt do at least a few completely boneheaded things at one point in their youth.

[-] LallyLuckFarm@beehaw.org 24 points 5 months ago

when reached for comment, the makers of the ai tool stated that they were shocked that the tool was used for the purpose for which it was built

/s because it's not a real quote but also:

~~/s~~

[-] schizanon@beehaw.org 13 points 5 months ago

I am so lucky I wasn't in school when AI was around.

[-] BruceTwarzen@lemm.ee 6 points 5 months ago

People used to say that about social media

[-] lowleveldata@programming.dev 8 points 5 months ago

Can't say I'm surprised

[-] autotldr 3 points 5 months ago

🤖 I'm a bot that provides automatic summaries for articles:

Click here to see the summary

In addition to probation, the teens will also be required to attend classes on gender and equality, as well as on the "responsible use of information and communication technologies," a press release from the Juvenile Court of Badajoz said.

In addition to mental health impacts, victims have reported losing trust in classmates who targeted them and wanting to switch schools to avoid further contact with harassers.

Minors targeting classmates may not realize exactly how far images can potentially spread when generating fake child sex abuse materials (CSAM); they could even end up on the dark web.

An investigation by the United Kingdom-based Internet Watch Foundation (IWF) last year reported that "20,254 AI-generated images were found to have been posted to one dark web CSAM forum in a one-month period," with more than half determined most likely to be criminal.

While lawmakers struggle to apply existing protections against CSAM to AI-generated images or to update laws to explicitly prosecute the offense, other more drastic solutions to prevent the harmful spread of deepfakes have been proposed.

Ars could not immediately reach Meta for comment on efforts to combat the proliferation of AI-generated CSAM on WhatsApp, the private messaging app that was used to share fake images in Spain.

Saved 72% of original text.

load more comments

this post was submitted on 10 Jul 2024

171 points (100.0% liked)

Technology

37800 readers

166 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago

MODERATORS

TheRtRevKaiser@beehaw.org

alyaza@beehaw.org

gyrfalcon@beehaw.org

SemioticStandard@beehaw.org

coldredlight@beehaw.org

rs5th@beehaw.org

TheRtRevKaiser@kbin.social

remington@beehaw.org