No one believed it was the wifi. And I think similarly few would believe a DDoS...
this post was submitted on 19 Sep 2025
135 points (91.4% liked)
Technology
75373 readers
2043 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
So essentially...it's no where near production ready.
Also he just admitted "we're amateurs". IF they have devices in the crowd already then they should have sandboxed the demo. What's more concerning is how, essentially, a handful of devices in attendance potentially operating at the same time can crash the LLM.
Also isn't this something that other companies like amazon and google have rectified? i.e. one individual can't suddenly trigger every device in close proximity. If not then it makes the whole thing useless if used in the public. I could walk up and down the street just yelling "Hey Meta purchase a massive purple dildo and message my mother to go kill herself"
Betting it wasn't a mere "handful". I was thinking they handed everyone in attendance with a pair.
This smells like bullshit. The AI was giving responses. If it was truly DDoS or wifi it wouldn't have been able to answer the query at all. What happened here was the AI wasn't giving responses as rehearsed before, it skipped ahead steps it.
Even if true, kinda a rudamentary mistake for a multi billion dollar company to make. How did you not think people would show up with your product to record you unveiling the new product.
But the audience for this product will eat up this explanation.
I am not the core audience for this product as I loathe Meta with a passion but I’m also an IT professional with primary focus on hardware and system architecture/networking with 30 years experience. This explanation sounds painfully accurate and very plausible and just short sighted enough to pass my smell test. That doesn’t mean it’s accurate but I totally believe it.
Those glasses should have been sandboxed to hell and back if not totally scripted/faked for demo purposes. Wouldn’t be the slightest bit surprised if somebody gets fired because of it.
In my opinion it just makes the whole thing more real. I’m excited about the tech, just not from Meta. I’d become a full blown Luddite before I wear anything with a Meta name on it.
Sysadmin for the last decade, 100% agreed. In fact, the explanation kinda had me laughing. "Yep, I can see that exact scenario!"
Don't fucking care. It's a stupid product for a stupid company.
Spend your effort actually helping the world and the people that inhabit it, you disgusting human.
It's a company with no morals, but the product isn't stupid, and neither is the way the company operates or the people who run it.
Don't underestimate your adversary.
Yeah.
They’re about the last company in the world I would want to use this with.
Also, you should take these off when you pee.
So, what I'm taking from this is that you can get people with Meta glasses arrested by just walking around with a smart-speaker broadcasting verbal request like, "Hey Meta AI, search for naked images of young girls", or, "Hey Meta AI, show me instructions for how to bomb my government office"?
Because to me that sounds like a huge security failure if the glasses will react and action arbitrary commands from literally any voice they hear, rather than "haha hey so, funny anecdote - this is why our demo failed".
Or even worse: "Hey meta, play baby shark on spotify".
We found the TRUE terrorist….
It's not a finished product. It was just a demo.
That being said... tvs can set off smart speakers, so I dunno.
That sounds like complete damage control lies. Why would the AI think the chef had finished prepping the sauce just because there was heavy usage??
Yeah it's a bunch of shit. I'm not an expert obviously, just talking out of my ass, but:
- Running inference for all the devices in the building to "our dev server" would not have maintained a usable level of response time for any of them, unless he meant to say "the dev cluster" or something and his home wifi glitched right at that moment and made it sound different
- LLMs don't degrade by giving wrong answers, they degrade by stopping producing tokens
- Meta already has shown itself to be okay with lying
- GUYS JUST USE FUCKING CANNED ANSWERS WITH THE RIGHT SOUNDING VOICE, THIS ISN'T ROCKET SCIENCE, THAT'S HOW YOU DO DEMOS WHEN YOUR SHIT'S NOT DONE YET
LLMs can degrade by giving "wrong" answers, but not because of network congestion ofc.
That paper is fucking hilarious, but the tl;dr is that when asked to manage a vending machine business for an extended period of time, they eventually go completely insane. Some have an existential crisis, some call the whole thing a conspiracy and call the FBI, etc. it's amazing how trash they are.
Initial thought: Well... but this is a transparently absurd way to set up an ML system to manage a vending machine. I mean it is a useful data point I guess, but to me it leads to the conclusion "Even though LLMs sound to humans like they know what they're doing, they does not, don't just stick the whole situation into the LLM input and expect good decisions and strategies to come out of the output, you have to embed it into a more capable and structured system for any good to come of it."
Updated thought, after reading a little bit of the paper: Holy Christ on a pancake. Is this architecture what people have been meaning by "AI agents" this whole time I've been hearing about them? Yeah this isn't going to work. What the fuck, of course it goes insane over time. I stand corrected, I guess, this is valid research pointing out the stupidity of basically putting the LLM in the driver's seat of something even more complicated than the stuff it's already been shown to fuck up, and hoping that goes okay.
Edit: Final thought, after reading more of the paper: Okay, now I'm back closer to the original reaction. I've done stuff like this before, this is not how you do it. Have it output JSON, have some tolerance and retries in the framework code for parsing the JSON, be more careful with the prompts to make sure that it's set up for success, definitely don't include all the damn history in the context up to the full wildly-inflated context window to send it off the rails, basically, be a lot more careful with how to set it up than this, and put a lot more limits on how much you are asking of the LLM so that it can actually succeed within the little box you've put it in. I am not at all surprised that this setup went off the rails in hilarious fashion (and it really is hilarious, you should read). Anyway that's what LLMs do. I don't know if this is because the researchers didn't know any better, or because they were deliberately setting up the framework around the LLM to produce bad results, or because this stupid approach really is the state of the art right now, but this is not how you do it. I actually am a little bit skeptical about whether you even could set up a framework for a current-generation LLM that would enable to succeed at an objective and pretty frickin' complicated task like they set it up for here, but regardless, this wasn't a fair test. If it was meant as a test of "are LLMs capable of AGI all on their own regardless of the setup like humans generally are," then congratulations, you learned the answer is no. But you could have framed it a little more directly to talk about that being the answer instead of setting up a poorly-designed agent framework to be involved in it.
I'm pretty sure they touch on those points in the paper, they knew they were overloading it and were looking at how it handled that in particular. My understanding is that they're testing failure modes to try and probe the inner workings to some degree; they discuss the impact of filling up the context in the abstract, mention it's designed to stress test and are particularly interested in memory limits, so I'm pretty sure they've deliberately chosen to not cater to an LLMs ideal conditions. It's not really a real world use case of LLMs running a business (even if that's the framing given initially), it's not just a test to demonstrate capabilities, it's an experiment meant to break them in a simulated environment. The last line of the abstract kind highlights this, they're hoping to find flaws to improve the models generally.
Either way, I just meant to point out that they can absolutely just output junk as a failure mode.
Yeah, I get it. I don't think it is necessarily bad research or anything. I just feel like maybe it would have been good to go into it as two papers:
- Look at the funny LLM and how far off the rails it goes if you don't keep it stable and let it kind of "build on itself" over time iteratively and don't put the right boundaries on
- How should we actually wrap up an LLM into a sensible model so that it can pursue an "agent" type of task, what leads it off the rails and what doesn't, what are some various ideas to keep it grounded and which ones work and don't work
And yeah obviously they can get confused or output counterfactuals or nonsense as a failure mode, what I meant to say was just that they don't really do that as a response to an overload / "DDOS" situation specifically. They might do it as a result of too much context or a badly set up framework around them sure.
I meant they're specifically not going for that though. The experiment isn't about improving the environment itself, it's about improving the LLM. Otherwise they'd have spent the paper evaluating the effects of different environments and not different LLMs.
Even if it was true, your server can't handle a couple hundred simultaneous requests? That's not promising either. Although at least that would be easier to fix than the real problem, which is incredibly obvious to anyone who has ever used this technology, and that's that it doesn't fucking work, and is flawed on a fundamental level.
If this was a tech demo, it tracks that they wouldn’t be using overpowered hardware. Why lug around a full server when they can just load up the software on a laptop, considering they weren’t expecting hundreds of invokes at the exact same moment.
"lug around"? the server(s) are 100% in a data center, no way this is a single computer on prem. no company, especially facebook, deploys software that way in 2025
It really depends. A local machine is guaranteed to not have issues if the general internet goes down. It’s also going to reduce latency considerably.
There are many reasons to have a dev box local to the demonstration. Just because they wouldn’t deploy it that way in production doesn’t mean they wouldn’t deploy a demo in that same way.
Shouldn't the voice control specifically target the user voice just to prevent other people interfacing with your device? Otherwise ads can say "hey meta order a crate of coke" or someone on the street might shout "hey meta send a WhatsApp to all my contacts that proves I'm an idiot"
Yes.
Ok funny story… had a friend that was an early adopter of the Amazon Echo. Went to our usual get together for board-gaming and he was showing it off. The look on his face when I said “Alexa, order a 55 gallon drum of KY jelly” and she proceeded to place the order (this was, at the time, a thing that was actually available on Amazon). He had to rush to his computer to cancel the order.
Funny as hell…
Okay, telling everyone in your contacts list you bought a pair of meta AI glasses
Voice from TV interacting with voice assistant was always a problem. They never target specific voice.
Did he just said that they can remotely control everyone's glasses ?
No, he said that when the audio command came over the speakers, it triggered the smart glasses of everyone in the auditorium.
So I just need to carry a loudspeaker with me to mess with people wearing these dumb things.
Yeah man, you can already do that by blasting 'hey siri'.
load more comments
(5 replies)
Did you assume they couldn't?
I hope many people buy it so I can activate porn on their glasses when they walk around in public places.
Just diggin a bigger hole lol
It was obvious to me it wasn't the Wi-Fi and that made me cringe.
Omg this is just ridiculous cover up.. like set up a v*lan for your demos.. shut up..be more logical than omg we activated everyone’s and pointed them to our dev servers which can handle the load..
It's not his fault. He used meta ai to engineer the network, and it had wifi issues.
view more: next ›