206

Google's call-scanning AI could dial up censorship by default, privacy experts warn (techcrunch.com)

submitted 6 months ago by ForgottenFlux@lemmy.world to c/technology@lemmy.world

30 comments fedilink hide all child comments

A feature Google demoed at its I/O confab yesterday, using its generative AI technology to scan voice calls in real time for conversational patterns associated with financial scams, has sent a collective shiver down the spines of privacy and security experts who are warning the feature represents the thin end of the wedge. They warn that, once client-side scanning is baked into mobile infrastructure, it could usher in an era of centralized censorship.

Apple abandoned a plan to deploy client-side scanning for CSAM in 2021 after a huge privacy backlash. However, policymakers have continued to heap pressure on the tech industry to find ways to detect illegal activity taking place on their platforms. Any industry moves to build out on-device scanning infrastructure could therefore pave the way for all-sorts of content scanning by default — whether government-led or related to a particular commercial agenda.

Meredith Whittaker, president of the U.S.-based encrypted messaging app Signal, warned: “This is incredibly dangerous. It lays the path for centralized, device-level client side scanning.

“From detecting ‘scams’ it’s a short step to ‘detecting patterns commonly associated w[ith] seeking reproductive care’ or ‘commonly associated w[ith] providing LGBTQ resources’ or ‘commonly associated with tech worker whistleblowing.’”

all 31 comments

sorted by: hot top controversial new old

[-] Kolanaki@yiffit.net 35 points 6 months ago

If it starts being mandatory and not an optional feature you can turn on or off at will, I'll worry.

[-] dukethorion@lemmy.world 49 points 6 months ago

You can turn it off all you want, it won't stop the scanning and collection.

Like turning off Google Location history, its just hidden from your view.

[-] lud@lemm.ee 17 points 6 months ago

Like turning off Google Location history, its just hidden from your view.

I obviously don't trust Google (who does?) but you say that like it's a verifiable fact. Is it?

[-] TheGrandNagus@lemmy.world 4 points 5 months ago

IIRC it was, then the EU stopped it. I don't know if they still do it outside of the EU.

[-] muntedcrocodile@lemm.ee 4 points 5 months ago

Yep put ur phone traffic through wire shark and u really location gets sent to google regardless. They do claim not to store it tho but I wouldn't believe that for a second.

[-] lud@lemm.ee 0 points 5 months ago

Yep put ur phone traffic through wire shark and u really location gets sent to google regardless..

Have you done that?

[-] muntedcrocodile@lemm.ee 2 points 5 months ago

https://stackoverflow.com/questions/9555403/capturing-mobile-phone-traffic-on-wireshark#9570093

[-] lud@lemm.ee -1 points 5 months ago

I take that as a no.

[-] dukethorion@lemmy.world 3 points 6 months ago

We don't know for sure. Google has never lied to us before, right?

[-] lud@lemm.ee -4 points 5 months ago

So no

[-] dukethorion@lemmy.world 1 points 5 months ago

Its okay if you work for Google, you can just say so.

Or you are naive and gullible.

[-] lud@lemm.ee -1 points 5 months ago

Lol, I'm not saying Google is or isn't doing that. I'm just saying that you are just spewing bullshit without any evidence whatsoever.

[-] Coasting0942@reddthat.com 12 points 6 months ago

Silly user: you can’t fully disable features intended to keep you safe

[-] LesserAbe@lemmy.world 9 points 6 months ago

Trouble is when everybody else uses it, just because you turn it off doesn't mean you won't have issues.

[-] retrospectology@lemmy.world 28 points 6 months ago* (last edited 6 months ago)

Combined with how easy it is becoming to create an AI copy of a person's voice you're pretty soon not going to ever be sure if what you're saying or hearing on a phone is actually what's being said or if it's being edited in real time. China's gonna love this shit.

Really hate tech bros who just keep recklessly pushing ahead on this stuff. Absolutely the worst scenario for AI.

[-] ChaoticNeutralCzech@feddit.de 12 points 6 months ago* (last edited 6 months ago)

The title feels weird. Is this really a way to use "dial up"?

== CISCO TM-33k6m2 DIAL-UP MODEM QUICK SETUP ==

STEP 2: SELECT PROVIDER PROFILE

  [1] AT&T
  [2] Comcast
  [3] AOL
  [4] Censorship (default)
  
  [0] Specify Provider Parameters...

> _

[-] Halosheep@lemm.ee 16 points 6 months ago

Yes, the phrase comes from turning a dial to a higher value. Think, for example, of a guitar amplifier. You could "dial up" the volume, meaning you are increasing the volume.

Dial up internet refers to a different kind of dialing.

[-] ChaoticNeutralCzech@feddit.de 1 points 6 months ago

Oh, I assumed it was another "witty" word association by a journalist.

[-] chunkystyles@sopuli.xyz 11 points 5 months ago

I'm not advocating for this, but I could see it effectively ending phone scams that often prey on the elderly.

[-] hasnt_seen_goonies@lemmy.world 2 points 5 months ago

Yeah, I don't want this for my phone right now, but I do think that this would help me sleep easier with my grandfather who has already fallen for multiple scams.

[-] captain_aggravated@sh.itjust.works 7 points 5 months ago

better idea: shut down the phone network entirely.

[-] desktop_user@lemmy.blahaj.zone 1 points 5 months ago

sadly there is too much legacy infrastructure to do that without huge problems. Also idiotic forms of two factor authentication exist.

[-] Infynis@midwest.social 6 points 6 months ago

You know that scifi trope where someone reaches into a bit of tech, and rips out a blinking tracker? Get ready to have to do that when you buy a new phone

[-] LesserAbe@lemmy.world 5 points 6 months ago* (last edited 6 months ago)

"Thin end of the wedge" is a good way to put it.

[-] AceFuzzLord@lemm.ee 4 points 5 months ago

I can imagine that in the worst case scenario, company xyz pays millions to gøøgle to have its phone call scamming AI look out for anything that could be taken as even the most mild criticism and cut off your call. That, or the AI randomly cuts out the call because it hallucinates you saying bad things.

[-] possiblylinux127@lemmy.zip 3 points 5 months ago

I'd be worried about the potential chilling effects. People are likely to self censor talking about topics that could "get them on a list"

[-] Mango@lemmy.world 0 points 5 months ago

Yes, please censor the spam.

this post was submitted on 16 May 2024

206 points (97.2% liked)

Technology

59287 readers

4019 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS