Technology

It gave me flashbacks when the Replit guy complained that the LLM deleted his data despite being told in all caps not to multiple times.

People really really don't understand how these things work...

In my experience it can, but it has been pretty uncommon. But I also don't usually ask questions with only one answer.

Also, generally the best interfaces for LLM will combine non-LLM facilities transparently. The LLM might be able to translate the prose to the format the math engine desires and then an intermediate layer recognizes a tag to submit an excerpt to a math engine and substitute the chunk with output from the math engine.

Even for servicing a request to generate an image, the text generation model runs independent of the image generation, and the intermediate layer combines them. Which can cause fun disconnects like the guy asking for a full glass of wine. The text generation half is completely oblivious to the image generation half. So it responds playing the role of a graphic artist dutifully doing the work without ever 'seeing' the image, but it assumes the image is good because that's consistent with training output, but then the user corrects it and it goes about admitting that the picture (that it never 'looked' at) was wrong and retrying the image generator with the additional context, to produce a similarly botched picture.

The only thing close to a decision that LLMs make is

That's not true. An "if statement" is literally a decision tree.

If you want to engage in a semantically argument, then sure, an “if statement” is a form of decision. This is a worthless distinction that has nothing to do with my original point and I believe you’re aware of that so I’m not sure what this adds to the actual meat of the argument?

The only reason they answer questions is because in the training data they’ve been provided

This is technically true for something like GPT-1. But it hasn't been true for the models trained in the last few years.

Okay, what was added to models trained in the last few years that makes this untrue? To the best of my knowledge, the only advancements have involved:

• Pre-training, which involves some additional steps to add to or modify the initial training data
• Fine-tuning, which is additional training on top of an existing model for specific applications.
• Reasoning, which to the best of my knowledge involves breaking the token output down into stages to give the final output more depth.
• “More”. More training data, more parameters, more GPUs, more power, etc.

I’m hardly an expert in the field, so I could have missed plenty, so what is it that makes it “understand” that a question needs to be answered that doesn’t ultimately go back to the original training data? If I feed it training data that never involves questions, then how will it “know” to answer that question?

it knows from its training data that sometimes accusations are followed by language that we interpret as an apology, and sometimes by language that we interpret as pushing back. It regurgitates these apologies without understanding anything, which is why they seem incredibly insincere

It has a large amount of system prompts that alter default behaviour in certain situations. Such as not giving the answer on how to make a bomb. I'm fairly certain there are catches in place to not be overly apologetic to minimize any reputation harm and to reduce potential "liability" issues.

System prompts are literally just additional input that is “upstream” of the actual user input, and I fail to see how that changes what I said about it not understanding what an apology is, or how it can be sincere when the LLM is just spitting out words based on their statistical relation to one another?

An LLM doesn’t even understand the concept of right or wrong, much less why lying is bad or when it needs to apologize. It can “apologize” in the sense that it has many examples of apologies that it can synthesize into output when you request one, but beyond that it’s just outputting text. It doesn’t have any understanding of that text.

And in that scenario, yes I'm being gaslite because a human told it to.

Again, all that’s doing is adding additional words that can be used in generating output. It’s still just generating text output based on text input. That’s it. It has to know it’s lying or being deceitful in order to gaslight you. Does the text resemble something that can be used to gaslight you? Sure. And if I copy and pasted that from ChatGPT that’s what I’d be doing, but an LLM doesn’t have any real understanding of what it’s outputting so saying that there’s any intent to do anything other than generate text based on other text is just nonsense.

There is no thinking

Partially agree. There's no "thinking" in sentient or sapient sense. But there is thinking in the academic/literal definition sense.

Care to expand on that? Every definition of thinking that I find involves some kind of consideration or reflection, which I would argue that the LLM is not doing, because it’s literally generating output based on a complex system of weighted parameters.

If you want to take the simplest definition of “well, it’s considering what to output and therefore that’s thought”, then I could argue my smart phone is “thinking” because when I tap on a part of the screen it makes decisions about how to respond. But I don’t think anyone would consider that real “thought”.

There are no decisions

Absolutely false. The entire neural network is billions upon billions of decision trees.

And a logic gate “decides” what to output. And my lightbulb “decides” whether or not to light up based on the state of the switch. And my alarm “decides” to go off based on what time I set it for last night.

My entire point was to stop anthropomorphizing LLMs by describing what they do as “thought”, and that they don’t make “decisions” in the same way humans do. If you want to use definitions that are overly broad just to say I’m wrong, fine, that’s your prerogative, but it has nothing to do with the idea I was trying to communicate.

The more we anthropomorphize these statistical text generators, ascribing thoughts and feelings and decision making to them, the less we collectively understand what they are

I promise you I know very well what LLMs and other AI systems are. They aren't alive, they do not have human or sapient level of intelligence, and they don't feel. I've actually worked in the AI field for a decade. I've trained countless models. I'm quite familiar with them.

Cool.

But "gaslighting" is a perfectly fine description of what I explained. The initial conditions were the same and the end result (me knowing the truth and getting irritated about it) were also the same.

Sure, if you wanna ascribe human terminology to what marketing companies are calling “artificial intelligence” and further reinforcing misconceptions about how LLMs work, then yeah, you can do that. If you care about people understanding that these algorithms aren’t actually thinking in the same way that humans do, and therefore believing many falsehoods about their capabilities, like I do, then you’d use different terminology.

It’s clear that you don’t care about that and will continue to anthropomorphize these models, so… I guess I’m done here.