So seeing the reaction on lesswrong to Eliezer's book has been interesting. It turns out, even among people that already mostly agree with him, a lot of them were hoping he would make their case better than he has (either because they aren't as convinced as him, or they are, but were hoping for something more palatable to the general public).
This review (lesswrong discussion here), calls out a really obvious issue: Eliezer's AI doom story was formed before Deep Learning took off, and in fact was mostly focusing on more GOFAI than neural networks, yet somehow, the details of the story haven't changed at all. The reviewer is a rationalist that still believes in AI doom, so I wouldn't give her too much credit, but she does note this is a major discrepancy from someone that espouses a philosophy that (nominally) features a lot of updating your beliefs in response to evidence. The reviewer also notes that "it should be illegal to own more than eight of the most powerful GPUs available in 2024 without international monitoring" is kind of unworkable.
This reviewer liked the book more than they expected to, because Eliezer and Nate Soares gets some details of the AI doom lore closer to the reviewer's current favored headcanon. The reviewer does complain that maybe weird and condescending parables aren't the best outreach strategy!
This reviewer has written their own AI doom explainer which they think is better! From their limited description, I kind of agree, because it sounds like the focus on current real world scenarios and harms (and extrapolate them to doom). But again, I wouldn't give them too much credit, it sounds like they don't understand why existential doom is actually promoted (as a distraction and source of crit-hype). They also note the 8 GPUs thing is batshit.
Overall, it sounds like lesswrongers view the book as an improvement to the sprawling mess of arguments in the sequences (and scattered across other places like Arbital), but still not as well structured as they could be or stylistically quite right for a normy audience (i.e. the condescending parables and diversions into unrelated science-y topics). And some are worried that Nate and Eliezer's focus on an unworkable strategy (shut it all down, 8 GPU max!) with no intermediate steps or goals or options might not be the best.
I hope somebody said: 'If we let Eliezer write this, everybody dies'
I have not read it, but the 8 gpu max thing is funny. I'm halfway there already. Worse perhaps, I'm not sure there still is a Voodoo Graphics card in a box somewhere (don't think those count as a gpu however).
And it is funny that microsoft will now be limited to 8 gpus. But your anarchist polycule commune with 10 people can have 80 of them.
see, this also includes ~~aella's bday party location~~ ~~rationalist castle~~ ~~greater sfba area polycule~~ imperial chinese harem where they will solve alignment in no time. all according to the plan
Hey, it was Caroline Ellison who wanted to be part of the Imperial Chinese Harem! Also, judging by Dragon Army and Leverage Research. the communes aren't anarchist! They are tightly controlled by a single great man type leader.
Doubt they can resist the urge to wrap those communes into contracts and so theyvwill be organisations
Yeah, in some hypothetical bizarro universe where they get their 8 (cutting edge) GPU limit actually passed, I bet the monitoring scheme would be loose in a way that Microsoft just has a box to tick while strict enough that it imposes untenable costs on private individuals.
lol they would just get a waiver from Hegseth that they're doing very important national security stuff with it
waiver that says: I can do whatever I want, possibly acquired with help of agent Jack Daniels
Also, all this would do is change the processing from GPU to CPU. Microsoft commissions AMD, Nvidia or Intel to create a technically-not-a-GPU CPU and just have a computer that uses GDDR instead of the standard DDR.
Given the USA legislature's incompetence, I imagine they would leave some sort of massive loopholes. Depending on the exact wording, you could get around it with technically not GPUs (as you suggest), or subdividing companies so each subdivision can be technically under the limit, or cranking up the size of individual GPUs so 8 GPUs is a massive amount of compute. Of course, I really doubt it would get that far in the first place, look at how they killed California's attempt at the most moderate AI legislation.
From what I've read (granted from other reviews), the limit would be the equivalent performance of 8 4090s. Which means (assuming we believe Nvidia's claims of 3352 AI TOPS for the 5090 vs the 1321 AI TOPS for the 4090) that you couldn't possess more than the equivalent of 3 5090s. Then that keeps going, so.
BREAKING: Police perform 5th anti GPU raid in as many days. Found another illegal weed farm instead. Is the tactic of checking for unusual power usage failing? Our digital AI of John McAfee expresses concern this might hurt bitcoin miners.