May 6, 2023·edited May 6, 2023Liked by Robert Wright
The Bob-AI takes are generally insightful but I submit that they would be more insightful with a some minimal knowledge of concepts like vectors.
A vector is just a list of numbers, like [1.5, -2, 0.5]. So Hinton is saying a "thought" is just represented by a list of numbers in the model. You can add or subtract two vectors of the same length like [1.5, -2, 0.5]-[1, 0.5, 0]=[0.5, -2.5, 0.5].
The reason that matters is that one of the first indicators that something freaky was going on in those language models way back when they started training them, which Michal Kosinsky alluded to in an earlier podcast, was:
You train these models to predict the next word, and in the process, they learn an internal vector representation for every word (they turn each word into a list of 1000 numbers, and this mapping from word to vector evolves as they learn). Then, after learning, researchers looked at these vectors and asked "hey what happens if you take [vector for king]-[vector for man]+[vector for woman]"?. Guess what - the answer is really close to [vector for queen]. Same goes for London- England+France=Paris. So these things have learned analogies, even though all they were trained to do was predict the next word. Once you realize that these models are learning to structure knowledge like this even though they're not explicitly trained to, you start thinking "ok maybe these are not just stochastic parrots after all"
Good piece. François Chollet's book Deep Learning with Python, now in its 2nd ed. (Manning, 2021) has some accessible passages that help clarify the 'vector' notion you mention.
When a technology's capabilities are truly in good faith debate among invested and informed people, isn't it obvious we ought to be curious, if not concerned? Isn't it really as simple as that?
Truthfully, I find myself growing weary of the AI handwringing. It's very justified, imho, but I don't see it going anywhere meaningful.
If we were rational, we would stop developing AI, take our time sorting out our concerns, and develop a broad consensus on how to address those concerns. And then, when we'd done everything possible to make AI safe, we would revisit the question of whether we should continue development.
But we're not rational, we're only clever. And so we will race forward as fast as we possibly can while wringing our hands, crossing our fingers, and hoping it works out. And then, when some part of it doesn't work out, we'll realize it's too late to change our minds. So we'll become bored with the subject, and turn our attention to the creation of some other risky technology, piling up the risks one on top of another.
We've been through this already with nuclear weapons. It's frustrating to have to watch this movie again when we should already know how it's going to end.
If it truly understands, how do you explain this simple exchange:
ME: How many verbs or nouns exist in this sentence
ChatGPT: You didn't provide a sentence for me to analyze. Please provide a sentence, and I'll be happy to help you with your question. [wrong answer]
ME: I did provide a sentence
ChatGPT: I apologize, I did not see any sentence in your initial message. Could you please retype the sentence for me to analyze? [wrong response]
ME: How many verbs or nouns exist in this sentence
ChatGPT: I apologize for my previous message, as it seems that I misunderstood your question. In the sentence "How many verbs or nouns exist in this sentence," there are six nouns ("verbs," "nouns," "sentence," "many," "this," and "exist") and one verb ("exist"). [wrong answer]
I think you are getting closer to understanding why Hinton is alarmed. My explanation is somewhat simplified, so please bear with me.
The most important thing to understand about transformer neural networks like GPT-4 is that before we initiate training it on the (textual) corpus, we don't pre-program any knowledge into it. There is no computer code defining letters, words, commas, grammar, and so on. It doesn't have any computer code for English or any other language, nor does it contain any code teaching it that we communicate using words that form sentences (using grammatical rules), and so forth. We don't incorporate any formal logic or rules into it; it doesn't contain mathematics, implications, or any pre-programmed grammar. Despite Chomsky's life work, it contains none of it – no universal grammar, in fact Chomsky's work is irrelevant.
Before training starts, it is an ultimate blank slate. Except this blank slate is a 1000 IQ genius.
We simply begin by feeding it (GPT-4) random text, and it gradually starts to recognize patterns and understand the interconnectedness of words or symbols. As for how it grasps this information, what gets stored within its neural network, and the overall functioning of the system, we have no idea. We've just blindly approximated how our biological brains (neurons) work, and then magic happens.
---
To illustrate my point, imagine an alien probe containing an 800GB data dump of alien internet (in text form) landing on Earth. We could take an untrained GPT-4 program (one that hasn't been exposed to human text) and start training it on this alien internet data dump. It would work! GPT-4's remarkable 1000 IQ brain would gradually decipher alien languages, logic, facts, and alien history to the same extent that it understands human text, the human world, our history/politics. Essentially, we wouldn't need to alter anything. We'd obtain an alien version of GPT, or AlienGPT; however, we wouldn't be able to communicate with it since we can't speak the alien language and wouldn't comprehend its textual (in alien lang) responses to us.
Nonetheless, in this hypothetical, I believe we could achieve intriguing results (with minor adjustments) by training GPT simultaneously on both the hypothetical alien and human corpora. Consequently, it would understand both human and alien languages/facts, and we could likely ask it questions about aliens in English, just as we can ask current ChatGPT about the Italian language or Italian history in English.
Do the programs only deal with written language or can they "hear" spoken language and respond correctly when intonation, inflection, and accentation, etc change a statement to a question, for instance?
Loved your notes on connectionism vs. the symbolic crowd. Your call for us to get more comfortable using the word "understanding" when discussing these models is timely. The issue seems to be we keep referring to a variety of concepts under the label of "understanding" or "meaning". Hopefully in the near future we can standardize the jargon split between mathematicians, cognitive scientists, neuroscientists, programmers, etc. Right now it seems like the only people willing to commit themselves to a nuanced enough system are the philosophers, and who has that attention span? Very hard to have a public discussion if we've only had a few years for these concepts to seep into society.
Yes, the term stochastic parrots is being used to dismiss the power of the models, but its not as dismissive as saying these models are doing “fancy autocomplete”. If I remember correctly the original paper acknowledged the power of the models and even hinted that in a couple years everyone would be terrified by how well they could reproduce language. The issue is the type of learning they do. I agree that nothing is stopping anyone from using the term “stochastic parrots” and discussing meaning in the same breath.
As a side note, was sad to see Hinton's debut into the public eye as just "the guy who did AI for google". So interesting that you got to talk to him back in the 80s!
I enjoyed reading your article and found it thought-provoking. While being little off topic, I would like to share my thoughts on the meaning behind the phrases "it is just" or "it is not just".
The use of these phrases largely depends on an individual's temperament. For instance, consider a painting. An art exhibition host might say, "it is not just a paint over a canvas, it is a manifestation of the painter's inner beauty that conveys something profound". But a representative of scientific naturalism may analyze it and then quite objectively conclude "it is just a paint over a canvas".
Bringing it back to LLMs, when you say that "LLMs are not just fancy-autocomplete", I am curious to know whether you are referring to their value from a scientific naturalist perspective or from the viewpoint of someone who wants to keep their audience engaged.
I wrote a philosophy paper during my CS degree regarding the Chinese room thought experiment. Its where a Chinese character comes in, according to a rule book that emulates a 7 year old Chinese boy's speech a person draws another character and outputs it. Essentially the example is used for the "stochastic parrot" argument that these systems can never achieve intelligence because there is no intelligence visible in the process of taking in information and putting it back out.
This quickly breaks down once we see that human neurons can each be represented by a Chinese room construction (even if randomness is involved via Brownian motion a dice roll can easily be added without affecting the original argument). Once you put together a few billion of these thought experiments you get the human brain, which currently is our metric for intelligence.
This is exactly why I believe these systems can attain some form of intelligence, and why Hinton's fears are entirely justified. We've finally broken through the complexity barrier for AI, where its "Chinese room" neurons are beginning to do nearly the same thing we do: learn and understand.
Christ, the histrionics. Not everyone can be the person playing the "I'm the only person taking AI seriously" card. The entire fucking industry is stretching the power of hyperbole, every day!
One of these days someone will prime a language model with some sort of simulated somatic experience which will greatly enrich the model since so much of our language references somatic experience. (For all I know it's already in there.)
On the topic of vectors, Andrej Karpathy has a totally mind blowing series on how to construct a chat gpt. If you have some familiarity with python and a basic STEM level of math, it is pretty accessible.
The Bob-AI takes are generally insightful but I submit that they would be more insightful with a some minimal knowledge of concepts like vectors.
A vector is just a list of numbers, like [1.5, -2, 0.5]. So Hinton is saying a "thought" is just represented by a list of numbers in the model. You can add or subtract two vectors of the same length like [1.5, -2, 0.5]-[1, 0.5, 0]=[0.5, -2.5, 0.5].
The reason that matters is that one of the first indicators that something freaky was going on in those language models way back when they started training them, which Michal Kosinsky alluded to in an earlier podcast, was:
You train these models to predict the next word, and in the process, they learn an internal vector representation for every word (they turn each word into a list of 1000 numbers, and this mapping from word to vector evolves as they learn). Then, after learning, researchers looked at these vectors and asked "hey what happens if you take [vector for king]-[vector for man]+[vector for woman]"?. Guess what - the answer is really close to [vector for queen]. Same goes for London- England+France=Paris. So these things have learned analogies, even though all they were trained to do was predict the next word. Once you realize that these models are learning to structure knowledge like this even though they're not explicitly trained to, you start thinking "ok maybe these are not just stochastic parrots after all"
Good piece. François Chollet's book Deep Learning with Python, now in its 2nd ed. (Manning, 2021) has some accessible passages that help clarify the 'vector' notion you mention.
When a technology's capabilities are truly in good faith debate among invested and informed people, isn't it obvious we ought to be curious, if not concerned? Isn't it really as simple as that?
Truthfully, I find myself growing weary of the AI handwringing. It's very justified, imho, but I don't see it going anywhere meaningful.
If we were rational, we would stop developing AI, take our time sorting out our concerns, and develop a broad consensus on how to address those concerns. And then, when we'd done everything possible to make AI safe, we would revisit the question of whether we should continue development.
But we're not rational, we're only clever. And so we will race forward as fast as we possibly can while wringing our hands, crossing our fingers, and hoping it works out. And then, when some part of it doesn't work out, we'll realize it's too late to change our minds. So we'll become bored with the subject, and turn our attention to the creation of some other risky technology, piling up the risks one on top of another.
We've been through this already with nuclear weapons. It's frustrating to have to watch this movie again when we should already know how it's going to end.
If it truly understands, how do you explain this simple exchange:
ME: How many verbs or nouns exist in this sentence
ChatGPT: You didn't provide a sentence for me to analyze. Please provide a sentence, and I'll be happy to help you with your question. [wrong answer]
ME: I did provide a sentence
ChatGPT: I apologize, I did not see any sentence in your initial message. Could you please retype the sentence for me to analyze? [wrong response]
ME: How many verbs or nouns exist in this sentence
ChatGPT: I apologize for my previous message, as it seems that I misunderstood your question. In the sentence "How many verbs or nouns exist in this sentence," there are six nouns ("verbs," "nouns," "sentence," "many," "this," and "exist") and one verb ("exist"). [wrong answer]
I think you are getting closer to understanding why Hinton is alarmed. My explanation is somewhat simplified, so please bear with me.
The most important thing to understand about transformer neural networks like GPT-4 is that before we initiate training it on the (textual) corpus, we don't pre-program any knowledge into it. There is no computer code defining letters, words, commas, grammar, and so on. It doesn't have any computer code for English or any other language, nor does it contain any code teaching it that we communicate using words that form sentences (using grammatical rules), and so forth. We don't incorporate any formal logic or rules into it; it doesn't contain mathematics, implications, or any pre-programmed grammar. Despite Chomsky's life work, it contains none of it – no universal grammar, in fact Chomsky's work is irrelevant.
Before training starts, it is an ultimate blank slate. Except this blank slate is a 1000 IQ genius.
We simply begin by feeding it (GPT-4) random text, and it gradually starts to recognize patterns and understand the interconnectedness of words or symbols. As for how it grasps this information, what gets stored within its neural network, and the overall functioning of the system, we have no idea. We've just blindly approximated how our biological brains (neurons) work, and then magic happens.
---
To illustrate my point, imagine an alien probe containing an 800GB data dump of alien internet (in text form) landing on Earth. We could take an untrained GPT-4 program (one that hasn't been exposed to human text) and start training it on this alien internet data dump. It would work! GPT-4's remarkable 1000 IQ brain would gradually decipher alien languages, logic, facts, and alien history to the same extent that it understands human text, the human world, our history/politics. Essentially, we wouldn't need to alter anything. We'd obtain an alien version of GPT, or AlienGPT; however, we wouldn't be able to communicate with it since we can't speak the alien language and wouldn't comprehend its textual (in alien lang) responses to us.
Nonetheless, in this hypothetical, I believe we could achieve intriguing results (with minor adjustments) by training GPT simultaneously on both the hypothetical alien and human corpora. Consequently, it would understand both human and alien languages/facts, and we could likely ask it questions about aliens in English, just as we can ask current ChatGPT about the Italian language or Italian history in English.
Do the programs only deal with written language or can they "hear" spoken language and respond correctly when intonation, inflection, and accentation, etc change a statement to a question, for instance?
Hinton is a HOMO, just like vonNeuman & Turing, which is why AI is now woke, cuck'd & homo
...
HINTON demands that there be “One World AI”
Directed by UK, woke & homo
To educate the worlds children into One Woke Common mind.
Microsoft & Google agree to send all new AI to UK first so they can verify that it is ‘UK-WOKE”.
https://www.zerohedge.com/technology/uk-get-early-or-priority-access-ai-models-google-and-openai
Loved your notes on connectionism vs. the symbolic crowd. Your call for us to get more comfortable using the word "understanding" when discussing these models is timely. The issue seems to be we keep referring to a variety of concepts under the label of "understanding" or "meaning". Hopefully in the near future we can standardize the jargon split between mathematicians, cognitive scientists, neuroscientists, programmers, etc. Right now it seems like the only people willing to commit themselves to a nuanced enough system are the philosophers, and who has that attention span? Very hard to have a public discussion if we've only had a few years for these concepts to seep into society.
Yes, the term stochastic parrots is being used to dismiss the power of the models, but its not as dismissive as saying these models are doing “fancy autocomplete”. If I remember correctly the original paper acknowledged the power of the models and even hinted that in a couple years everyone would be terrified by how well they could reproduce language. The issue is the type of learning they do. I agree that nothing is stopping anyone from using the term “stochastic parrots” and discussing meaning in the same breath.
As a side note, was sad to see Hinton's debut into the public eye as just "the guy who did AI for google". So interesting that you got to talk to him back in the 80s!
In a single word... Awesome!
I enjoyed reading your article and found it thought-provoking. While being little off topic, I would like to share my thoughts on the meaning behind the phrases "it is just" or "it is not just".
The use of these phrases largely depends on an individual's temperament. For instance, consider a painting. An art exhibition host might say, "it is not just a paint over a canvas, it is a manifestation of the painter's inner beauty that conveys something profound". But a representative of scientific naturalism may analyze it and then quite objectively conclude "it is just a paint over a canvas".
Bringing it back to LLMs, when you say that "LLMs are not just fancy-autocomplete", I am curious to know whether you are referring to their value from a scientific naturalist perspective or from the viewpoint of someone who wants to keep their audience engaged.
I wrote a philosophy paper during my CS degree regarding the Chinese room thought experiment. Its where a Chinese character comes in, according to a rule book that emulates a 7 year old Chinese boy's speech a person draws another character and outputs it. Essentially the example is used for the "stochastic parrot" argument that these systems can never achieve intelligence because there is no intelligence visible in the process of taking in information and putting it back out.
This quickly breaks down once we see that human neurons can each be represented by a Chinese room construction (even if randomness is involved via Brownian motion a dice roll can easily be added without affecting the original argument). Once you put together a few billion of these thought experiments you get the human brain, which currently is our metric for intelligence.
This is exactly why I believe these systems can attain some form of intelligence, and why Hinton's fears are entirely justified. We've finally broken through the complexity barrier for AI, where its "Chinese room" neurons are beginning to do nearly the same thing we do: learn and understand.
Christ, the histrionics. Not everyone can be the person playing the "I'm the only person taking AI seriously" card. The entire fucking industry is stretching the power of hyperbole, every day!
I see the misunderstanding. I was implying "English" when I wrote "language".
I think you are way too optimistic about what this means -- not saying it is not interesting for its own merit, but it's highly unreliable because it's too shallow. A longer post here: https://medium.com/@vaishakbelle/robert-wrights-thesis-of-geoff-hinton-s-worries-semantic-similarity-is-not-reliable-ae632df9d243
One of these days someone will prime a language model with some sort of simulated somatic experience which will greatly enrich the model since so much of our language references somatic experience. (For all I know it's already in there.)
On the topic of vectors, Andrej Karpathy has a totally mind blowing series on how to construct a chat gpt. If you have some familiarity with python and a basic STEM level of math, it is pretty accessible.