195
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 29 Jul 2023
195 points (98.5% liked)
Technology
37702 readers
302 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
What is a Chinese room?
Imagine that you're locked in a room. You don't know any Chinese, but you have a huge instruction book written in English that tells you exactly how to respond to Chinese writing. Someone outside the room slides you a piece of paper with Chinese writing on it. You can't understand it, but you can look up the characters in your book and follow the instructions to write a response.
You slide your response back out to the person waiting outside. From their perspective, it seems like you understand Chinese because you're providing accurate responses, but actually, you don't understand a word. You're just following instructions in the book.
Its a thought experiment involving a room where people write letters and shove them under the door of the Chinese kid's dorm room. He doesn't understand what's in the letters so he just forwards the mail randomly to his Russian and Indian neighbours who sometimes react angrily or happily depending on the content. Over time the Chinese kid learns which symbols make the Russian happy and which symbols make the Indian kid happy, and so forwards the mail correspondingly until he starts dating and gets a girlfriend that tells him that people really shouldn't be shoving mail under his door, and he shouldn't be forwarding mail he doesnt understand for free.
https://en.wikipedia.org/wiki/Chinese_room
Wow, solid wiki article! It's very hard to say anything on the subject that hasn't been said.
I didn't see the simple phrasing:
"What if the human brain is a Chinese Room?"
but that seems to fall under eliminative materialism replies.
Part of the Chinese Room program (both in our heads and in an AI) could be dedicated to creating the experience of consciousness.
Searle has no substantial logical reply to this criticism. He openly takes it on faith that humans have consciousness, which is funny because an AI could say the same thing.
The whole point of the Chinese room is that it doesn't need anything "dedicated to creating the experience of consciousness". It can pass the Turing test perfectly well without such a component. Therefore passing the Turing test - or any similar test based solely on algorithmic output - is not the same as possessing consciousness.
"The room understands" is a common counterargument, and it was addressed by Searle by proposing that a person memorize the contents of the book.
And the room passes the Turing test, that does not mean that "it passes all the tests we can throw at it". Here is one test that it would fail: it contains various components that respond to the word "red", but it does not contain any components that exclusively respond to any use of the word "red". This level of abstraction is part of what we mean by understanding. Internal representation matters.
Man, I love coming across terms like this.
Chinese Room, Chinese Walls, Dutch Treat, Dutch Uncle, Dutch Oven.
The Chinese room argument makes no sense to me. I cant see how its different from how young children understand and learn language.
My 2 year old sometimes unmistakable start counting when playing. (Countdown for lift off) Most numbers are gibberish but often he says a real number in the midst of it. He clearly is just copying and does not understand what counting is. At some point though he will not only count correctly but he will also be able to answer math questions. At what point does he “understand” at what point would you consider that chatgpt “understands”  There was this old tv programm where some then ai experts discussed the chinese room but they used a chinese restaurant for a more realistic setting. This ended with “So if i walk into a chinese restaurant, pick sm out on the chinese menu and can answer anything the waiter may ask, in chinese. Do i know or understand chinese? I remember the parties agreeing to disagree at that point.
Yes... the chinese experiment misses the point, because the Turing test was never really about figuring out whether or not an algorithm has "conscience" (what is that even?)... but about determining if an algorithm can exhibit inteligent behavior that's equivalent/indistinguishable from a human.
The chinese room is useless because the only thing it proves is that people don't know what conscience is, or what are they even are trying to test.
ChatGPT will never understand. LLMs have no capacity to do so.
To understand you need underlying models of real world truth to build your word salad on top of. LLMs have none of that.
What are your underlying models of the world built out of? Because I'm human, and mine are primarily built out of words.
How do you draw a line between knowing and understanding? Does a dog understand the commands it's been trained to obey?
Your underlying model is not made out of words, but out of concepts. You can have multiple words that all map to the same concept, i.e. cosmos, universe, space. Or a single word that map to different concepts.
No, they aren't. You represent them with words. But you sure as hell aren't responding to someone throwing you a football with words trying to figure out where it's going.
No, a dog (while many times more intelligent than chatGPT) doesn't understand anything.
https://thegradient.pub/othello/
LLMs are neural networks and are absolutely capable of understanding.
LLMs are criminally simplified neural networks at minimum thousands of orders less complex than a brain. Nothing we do with current neural networks resembles intelligence.
Nothing they do is close to understanding. The fact that you can train one exclusively on the rules of a simple game and get it to eventually infer a basic rule set doesn't imply anything like comprehension. It's simplistic pattern matching.
For one thing, understanding implies that a word is linked to a mental concept. So if you say "The car is red", you first need to mentally compare the mental concept of "red" to the car in question.
The Chinese room bypasses all of that, it can say "The car is red" without ever having seen a red object at all.
Do you maintain this line of reasoning if it only says “the car is red” when the car is in fact red. And is capable of changing the answer to correctly mentioned a different color when the item In question is a different question.
Some ai demos show that programs like gpt-4 are already way passed this when provided with, it can not only accurate describe whats in the image but also the context.
Some examples, mind these where shown in an openAI demo for gpt4, Open ai has not yet made their version of this tech publicly available.
When i see these examples, i am not convinced that the ai truly understands everything it is saying. But it does seem to understand context, One of the theories on how it can do this (they are still a black box) is talked about in some papers that large language models may actually create an internal model of the world similar to humans and use that for logical reasoning and context.
It doesn't matter if the answer is right. If the AI does not have an abstract understanding of "red" then it is using a different process to get to the answer than humans. And according to Searle, a Turing machine cannot have an abstract understanding of "red", no matter how complex the question or how complex an internal model is used to determine its answers.
Going back to the Chinese Room, it is possible that the instructions carried out by the human are based on a complex model. In fact, it is possible that the human is literally calculating the output of a trained neural net by summing the weights of nodes, etc. You could even carry out these calculations yourself, if you could memorize the parameters.
Your use of "black box" gets to the heart of it. Memorizing all of the parameters of a trained NN allows you to calculate an answer, but they don't give you any understanding what the answer means. And if they don't tell you anything about the meaning, then they don't tell the CPU doing that calculation anything about meaning either.
My gripe with the Chinese room is that Searle argues that his inability to understand Chinese means the program doesn't understand Chinese, but I could say the same thing about the human body.
The neurons that operate your vocal chords have no idea what they're saying, nor the ones in your hands any idea what they're writing, yet they can speak and write exactly because your brain tells them what to do. Your brain is exactly like that book as far as your mouth and hand neurons are concerned.
They don't need to understand language at all for your brain to be able to understand it and give instructions based on that understanding.
My only argument is at what point does an algorithm become sufficiently advanced that it is indistinguishable from a conscious being?
Because at the end of the day, most of what a brain does is information processing based on what it has previously learnt, and that's exactly what the algorithm is doing based on training data. A sufficient enough algorithm should surely be able to replicate understanding.
Sure, that isn't ChatGPT as we know it, as you can tell from its sometimes very zany responses that while it understands what words are valid responses, it doesn't understand what the words themselves mean, but we should reach that at some point, no?
Keep in mind ChatGPT is a language model. It's designed specifically to simulate sounding like a human. It does that... Okay. It doesn't understand the information or concepts it is using. It just sounds like it does. It can't reliably do basic maths and doesn't try or need to. It just needs to talk about it in a believably conversational way.
The brain does far more than process information. And ChatGPT doesn't even really do that.
While it's humorous how personally you are taking critiques of, chatGPT, it is unfortunate you are also demonstrating a fundamental lack of basic understanding of how ChatGPT works. Because of that, you have inflated what you believe chatGPT is doing.
Even when it gets basic maths wrong repeatedly. Because I can tell it 2+2=5 and it will agree with me. Conversationally. Since it has no concept of what 2+2=5 means.
Even though it has no memory of previous conversations, you believe it somehow retains understanding of concepts it discusses.
Even though it searches the internet to provide it the knowledge to answer questions, which is why it can cite sources that don't exist or don't support its claims, clearly demonstrating a fundamental lack of understanding the concept, or even the concept of citing sources.
Even though it was literally trained by humans telling it what the three most correct conversational response would be out of the 5 answers it gave every calibration question, you still believe it actually possesses intelligence above any human, who can have a conversation without making any of these mistakes.
I clearly put chatGPT "intelligence" as remarkably low as is possible, even non-existent. I also must concede in this situation it is smarter than at least one human I am aware of.
Well mostly the flaw is people assigning the test abilities it was never intended. Like testing intelligence. Turing outright as first thing in the paper presenting "imitation game" noted moving away from testing intelligence, since he didn't know to do that. Even on the realm of "testing intelligent kind of behavior" well more like human like behavior and human being here proxy for intelligent, it was mostly an academic research idea. Not a concrete test meant to be some milestone.
Turing wanted a way to step away from stuff like "thinking" and "intelligence" directly and then proposed "imitation game" mostly to the rest of the academia as way to develop computer systemics more towards "intelligent behavior". It was mostly like "hey we need some goal to have as a goal to have something to move towards with these intelligence things. This isn't intelligence, but it might be usefull goal or tool for development work". Since without some goal/project/aim to have project don't advance. So it was "how about we try to develop a thing, that can beat this imitation game. Wouldn't that be good stepping stone. Then we can move to the actual serious stuff. Just an idea".
However since this academic "thinking out aloud spitballing ideas" was uttered by the Alan Turing, it became the Turing Test and everyone started taking it way too seriously. Specially outside academia. Who yes did play the imitation game with their programs as it was intended as research and development tool.
exemplified by for example this little exerpt of "not trying to do anything too complete and ground breaking here":
It is pretty literally "I had a thought". Turin makes no claims of machine beating the game having any significance other than "machine beat this game I came up with, neat". There is no argument of if machine beats imitation game, then X or then it means Y is reached.
Rest of the paper is actually about objections to the core idea of "it could ever be possible for machine to think" and even as such said imitation game is kinda lead in or introduction to Turing's treatise various objections of various "it would be impossible for machine to think" arguments. Starting with theological argument of "only human soul can think. Hence no animal or machine can think." .... since it was 1950's.
I don’t understand how Chinese room is a valuable argument. To me, while the person inside the room doesn’t understand Chinese, the system room-person-instructions does. You don’t argue that you don’t understand your language because none of your individual neurons understand it.
I don’t claim that chatGPT “understands” the language, I just don’t think that this argument applies in general.
So? The room as a whole can speak chinese, what do i care how it works in the inside?
Also, can you give me a convincing argument that our brain doesn't work in essentially the same way?