So the guys who have been burning almost as much VC money as they have water and electricity in the name of building AGI have announced that they're totally gonna do it this time? Just a few more training runs man I swear this time we're totally gonna turn everyone into paperclips just let me have a few more runs.

Not gonna lie, "enforcing the line between ketchup and tomato sauce" isn't the sort of thing I'd expect the government to be into, but I guess I'm not mad about it?

Gotta be cheaper than buying new planes which would also have new engines. Generally there needs to be a pretty substantial increase in capability before it's worth retiring an existing platform, especially in a logistics role where you don't get as much benefit from the bleeding edge because nobody's supposed to be shooting at you in the first place.

I think the missing piece here is that B-52 isn't just a pretty good cargo hauler, it's a pretty good cargo hauler that we don't need to buy a whole new airframe to get. Think of it less as "we're commissioning these B-52s" and more as "hey look we found a way to use all these B-52s we already had" only this just keeps working forever.

It's not an exhaustive search technique, but it may be an effective heuristic if anyone is planning The Revolution(tm).

AI could be a viable test for bullshit jobs as described by Graeber. If the disinfotmatron can effectively do your job then doing it well clearly doesn't matter to anyone.

I mean, doesn't somebody still need to validate that those keys only get to people over 18? Either you have a decentralized authority that's more easily corrupted or subverted or else you have the same privacy concerns at certificate issuance rather than at time of site access.

I mean, the whole point of declaring this era post-truth is that these people have basically opted out of consensus reality.

Why don't they just hire a wizard to cast an anti-tiktok spell over all of Australia instead? It would be just as workable and I know a guy who swears he can do it for cheaper than whatever server costs they're gonna try and push.

Okay apparently it was my turn to subject myself to this nonsense and it's pretty obvious what the problem is. As far as citations go I'm gonna go ahead and fall back to "watching how a human toddler learns about the world" which is something I'm sure most AI researchers probably don't have experience with as it does usually involve interacting with a woman at some point.

In the real examples that he provides, the system isn't "picking up the wrong goal" as an agent somehow. Instead it's seeing the wrong pattern. Learning "I get a pat on the head for getting to the bottom-right-est corner of the level" rather than "I get a pat on the head when I touch the coin." These are totally equivalent in the training data, so it's not surprising that it's going with the simpler option that doesn't require recognizing "coin" as anything relevant. This failure state is entirely within the realms of existing machine learning techniques and models because identifying patterns in large amounts of data is the kind of thing they're known to be very good at. But there isn't any kind of instrumental goal establishing happening here as much as the system is recognizing that it should reproduce games where it moves in certain ways.

This is also a failure state that's common in humans learning about the world, so it's easy to see why people think we're on the right track. We had to teach my little on the difference between "Daddy doesn't like music" and "Daddy doesn't like having the Blaze and the Monster Machines theme song shout/sang at him when I'm trying to talk to Mama." The difference comes in the fact that even as a toddler there's enough metacognition and actual thought going on that you can help guide them in the right direction, rather than needing to feed them a whole mess of additional examples and rebuild the underlying pattern.

And the extension of this kind of pattern misrecognition into sci-fi end of the world nonsense is still unwarranted anthropomorphism. Like, we're trying to use evidence that it's too dumb to learn the rules of a video game as evidence that it's going to start engaging in advanced metacognition and secrecy.

~~Brief~~ overlapping thoughts between parenting and AI nonsense, presented without editing.

The second L in LLM remains the inescapable heart of the problem. Even if you accept that the kind of "thinking" (modeling based on input and prediction of expected next input) that AI does is closely analogous to how people think, anyone who has had a kid should be able to understand the massive volume of information they take in.

Compare the information density of English text with the available data on the world you get from sight, hearing, taste, smell, touch, proprioception, and however many other senses you want to include. Then consider that language is inherently an imperfect tool used to communicate our perceptions of reality, and doesn't actually include data on reality itself. The human child is getting a fire hose of unfiltered reality, while the in-training LLM is getting a trickle of what the writers and labellers of their training data perceive and write about. But before we get just feeding a live camera and audio feed, haptic sensors, chemical tests, and whatever else into a machine learning model and seeing if it spits out a person, consider how ambiguous and impractical labelling all that data would be. At the very least I imagine the costs of doing so are actually going to work out to be less efficient than raising an actual human being and training them in the desired tasks.

Human children are also not immune to "hallucinations" in the form of spurious correlations. I would wager every toddler has at least a couple of attempts at cargo cult behavior or inexplicable fears as they try to reason a way to interact with the world based off of very little actual information about it. This feeds into both versions of the above problem, since the difference between reality and lies about reality cannot be meaningfully discerned from text alone and the limited amount of information being processed means any correction is inevitably going to be slower than explaining to a child that finding a "Happy Birthday" sticker doesn't immediately make it their (or anyone else's) birthday.

Human children are able to get human parents to put up with their nonsense ny taking advantage of being unbearably sweet and adorable. Maybe the abundance of horny chatbots and softcore porn generators is a warped fun house mirror version of the same concept. I will allow you to fill in the joke about Silicon Valley libertarians yourself.

IDK. Felt thoughtful, might try to organize it on morewrite later.

First and foremost, the dunce is incapable of valuing knowledge that they don't personally understand or agree with. If they don't know something, then that thing clearly isn't worth knowing.

There is a corollary to this that I've seen as well, and it dovetails with the way so many of these guys get obsessed with IQ. Anything they can't immediately understand must be nonsense not worth knowing. Anything they can understand (or think they understand) that you don't is clearly an arcane secret of the universe that they can only grasp because of their innate superiority. I think that this is the combination that explains how so many of these dunces believe themselves to be the ubermensch who must exercise authoritarian power over the rest of us for the good of everyone.

See also the commenter(s) on this thread who insist that their lack of reading comprehension is evidence that they're clearly correct and are in no way part of the problem.