1291
"prompt engineering"
(lemmy.world)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 10 Apr 2024
1291 points (99.0% liked)
Programmer Humor
19572 readers
748 users here now
Welcome to Programmer Humor!
This is a place where you can post jokes, memes, humor, etc. related to programming!
For sharing awful code theres also Programming Horror.
Rules
- Keep content in english
- No advertisements
- Posts must be related to programming or programmer topics
founded 1 year ago
MODERATORS
What I think is amazing about LLMs is that they are smart enough to be tricked. You can't talk your way around a password prompt. You either know the password or you don't.
But LLMs have enough of something intelligence-like that a moderately clever human can talk them into doing pretty much anything.
That's a wild advancement in artificial intelligence. Something that a human can trick, with nothing more than natural language!
Now... Whether you ought to hand control of your platform over to a mathematical average of internet dialog... That's another question.
It's not. Whenever someone talks about how LLMs are just statistics, ignore them unless you know they are experts. One thing that convinces me that ANNs really capture something fundamental about how human minds work is that we share the same tendency to spout confident nonsense.
It literally is just statistics... wtf are you on about. It's all just weights and matrix multiplication and tokenization
Well on one hand yes, when you're training it your telling it to try and mimic the input as close as possible. But the result is still weights that aren't gonna reproducte everything exactly the same as it just isn't possible to store everything in the limited amount of entropy weights provide.
In the end, human brains aren't that dissimilar, we also just have some weights and parameters (neurons, how sensitive they are and how many inputs they have) that then output something.
I'm not convinced that in principle this is that far from how human brains could work (they have a lot of minute differences but the end result is the same), I think that a sufficiently large, well trained and configured model would be able to work like a human brain.
Not an LLM specifically, in particular lack of backtracking and the network depth limits as well as interconnectivity limits sets hard limits on capabilities.
https://www.lesswrong.com/posts/XNBZPbxyYhmoqD87F/llms-and-computation-complexity
https://garymarcus.substack.com/p/math-is-hard-if-you-are-an-llm-and
https://arxiv.org/abs/2401.11817
https://www.marktechpost.com/2023/08/01/this-ai-research-dives-into-the-limitations-and-capabilities-of-transformer-large-language-models-llms-empirically-and-theoretically-on-compositional-tasks/?amp
Humans have a completely different memory model and a in large part a very different way of linking together learned concepts to form their world view and to develop interdisciplinary skills, allowing us to solve many kinds of highly complex tasks as long as we can keep enough of it in our memory.
See, none of these is statistics, as such.
Weights is maybe closest but they are supposed to represent the strength of a neural connection. This is originally inspired by neurobiology.
Matrix multiplication is linear algebra and encountered in lots of contexts.
Tokenization is a thing from NLP. It's not what one would call a statistical method.
So you can see where my advice comes from.
Certainly there is nothing here that implies any kind of averaging going on.
If there's no averaging, why do they repeat stereotypes so often?
Why would averaging lead to repetition of stereotypes?
Anyway, it's hard to say LLMs output what they do. GPTisms may have to do with the system prompt or they may result from the fine-tuning. Either way, they don't seem very internet average to me.
The TLDR is that pathways between nodes corresponding to frequently seen patterns (stereotypical sentences) gets strengthened more than others and therefore it becomes more likely that this pathway gets activated over others when giving the model a prompt. These strengths correspond to probabilities.
Have you seen how often they'll sign a requested text with a name placeholder? Have you seen the typical grammar they use? The way they write is a hybridization of the most common types of texts it has seen in samples, weighted by occurrence (which is a statistical property).
It's like how mixing dog breeds often results in something which doesn't look exactly like either breed but which has features from every breed. GPT/LLM models mix in stuff like academic writing, redditisms and stackoverflowisms, quoraisms, linkedin-postings, etc. You get this specific dryish text full of hedging language and mixed types of formalisms, a certain answer structure, etc.
That's a) not how it works and b) not averaging.
A) I've not yet seen evidence to the contrary
B) you do know there's a lot of different definitions of average, right? The centerpoint of multiple vectors is one kind of average. The median of online writing is an average. The most common vocabulary, the most common sentence structure, the most common formulation of replies, etc, those all form averages within their respective problem spaces. It displays these properties because it has seen them so often in samples, and then it blends them.
I accidentally clicked reply, sorry.
I don't think that any definition applies to this. But I'm no expert on averages. In any case, the training data is not representative of the internet or anything. It's also not training equally on all data and not only on such text. What you get out is not representative of anything.
You don't need it to be an average of the real world to be an average. I can calculate as many average values as I want from entirely fictional worlds. It's still a type of model which favors what it sees often over what it sees rarely. That's a form of probability embedded, corresponding to a form of average.
Hmm. I'm not really sure why anyone would write such a text. There is no "weighted proportionality" (or pathways). Is this a common conception?
I guess you picked up on the fact that transformers output a probability distribution. I don't think anyone calls those an average, though you could have an average distribution. Come to think of it, before you use that to pick the next token, you usually mess with it a little to make it more or less "creative". That's certainly no longer an average.
You can see a neural net as a kind of regression analysis. I don't think I have ever heard someone calling that a kind of average, though. I'm also skeptical if you can see a transformer as a regression but I don't know this stuff well enough. When you train on some data more often than on other data, that is not how you would do a regression. Certainly, once you start RLHF training, you have left regression territory for good.
The GPTisms might be because they are overrepresented in the finetuning data. It might also be from the RLHF and/or brought out by the system prompt.
You should worry more about whether you have seen evidence that supports what you are saying. So, what kind of evidence do you want? A tutorial on coding neural nets? The math? Video or text?
Text explaining why the neural network representation of common features (typically with weighted proportionality to their occurrence) does not meet the definition of a mathematical average. Does it not favor common response patterns?