Great review! I was wondering if and how LLMs deal with punctuation? I would assume they could be assigned their own vector, but their meaning is far less concrete and operates primarily to *modify* the meaning of words and sentences. For example, there is a WORLD of difference between:
"Rachael Ray finds inspiration in cooking her family and her dog"
and
"Rachael Ray finds inspiration in cooking, her family, and her dog"
Any brief explanation of how punctuation and other language modifiers (accent marks for non-English language perhaps) works would be very welcome!
This is one of the aspects of LLMs that we simplified to keep the explainer manageable. The basic unit of analysis in an LLM isn't words but rather tokens. The most common words are their own tokens, but longer and less common words are often made up of multiple tokens.
So I'm pretty sure ChatGPT treats "," as its own token and represents it with a vector. As you say, punctuation largely affects sentences by changing the meaning of nearby words, so I assume that LLMs figure this out by having nearby words "notice" the comma via the attention mechanism. But I don't know of any specific research on this question.
I’m far from an expert, but I believe punctuation marks have their own tokens. As footnote 1 says, the “words” LLMs operate on are “tokens”, some of which are whole words and some are fragments — so “hasn't” may be broken into “has” and “n't”. And IIRC there are tokens for word separators like commas, periods, semicolons..l
As Tim wrote, we didn't write about tokenization, but technically, most LLMs represent input as a series of "tokens", as opposed to words. Before the LLM is trained, there's a *tokenizer* that is trained, which basically amounts to learning all the "tokens" of a language. One way of thinking about this is that all the alphabetic characters of a language ("a", "b", "c", etc.) and also punctuation (".", ",", etc.) start out as their own tokens. Then, gradually, the tokenizer learns which characters co-occur with which other characters, i.e., which it then "merges" into a newer token representation. So words like "cat" typically would be their own token. The number of tokens this process ultimately results in is constrained by the researchers, e.g., they decide they want only 50K tokens overall in the model's vocabulary. (There are also really interesting questions about to what extent the "tokens" a model learns overlap with basic morphemes, i.e., units of meaning, in a language.)
Returning to your specific example, the two sentences would indeed have different token-level representations.
Simplifying a bit***, the first would be something like:
As Tim noted, those additional "," tokens will change the model's representations of adjacent tokens, as well as predictions of upcoming words. To know exactly how that would work in this specific case——and to see how successfully GPT differentiates those meanings——we'd need to actually look at different examples involving commas and ambiguity.
***Technically, some of these words would likely be broken up into tokens of their own. E.g., perhaps "cooking" is broken up into "cook" and "ing". But we'd have to run them through the tokenizer to be sure.
Punctuation and accent marks are entirely different sorts of things. The latter are mostly just composite glyphs representing different phonemes (e.g. 'o' vs 'ö') which could just as well have been represented by completely different glyphs like most phonemes are (e.g. 'k' vs 'g').
The tokens are mostly an optimization to cut training time and make the context window bigger. LLMs work okay on untokenized text; even if they only have English tokens they can translate to Chinese pretty well.
Tim, great article! I’m wondering if we can translate your blog into Chinese and post it on AI community in China. We will highlight your name and keep the original link on the top of the translated version. Thank you.
Thanks for an awesome writeup. I have two questions:
1) Is the model "seeded" with certain rules in the Translators that are provided by humans? Like ... did *we* tell it what the difference is between a noun and a verb, or is that something that it learns from the training data itself?
2) How do the conclusions drawn from the Translators wind up back in the main vector space? Does the model need to know that "Warsaw" was correct to make an update to the "Poland" vector?
(1) No the weights of the transformers are initialized entirely at random. Everything it knows about natural language it learns "from scratch" by detecting patterns of words in its training set. (Yes this is crazy! Machine learning experts found it mind-blowing when it started happening ~5 years ago.)
(2) After the final layer outputs its word vectors, the model uses a fairly simple neural network that takes the final word vector as an input and produces a prediction for the next word. In the Poland/Warsaw example, the encoding that leads to a prediction of "Warsaw" would be encoded in the vector for ":"—the final "word" of the sequence. Nobody knows exactly how the model "encodes" the word Warsaw into this final vector, we just know that if we apply the decode network to the final vector in the input you get a next-word prediction that is usually correct.
What may happen instead is that OpenAI will use your conversations or some modified version as part of the corpus for future training runs or as some other sort of input into future GPT versions. You can imagine some sort of RLHF-like process that uses conversations to fine-tune output.
I forgot to mention something else: Each time you submit text to ChatGPT, *all* of the previous text in the conversation is submitted to the model as well.
As explained in the article, these models work as next-word predictors and the next word is predicted based upon all the previous words in your conversation.
But the conversations you have are distinct from each other.
One thing to understand is that these models can only "reason" about a (relatively) small amount of text. In the case of GPT-3 around 4k words. So, if your conversation with GPT-3 extends beyond this number of words...including the answers the model has given you...the oldest words begin getting cut off.
Excellent and incisive explanatory piece. Great use of visual analogues to explain the power of computer science, as well as social science to elucidate consequences of these models.
In my opinion , the intuitive reason why Feed Forward part is more parameter heavy is because it is doing heavy lifting. If we are given a piece of text , and if we are asked to find contexts of words (attention) , on the basis of trained model in our brain , we could do it really fast , but then we are asked to predict the next word in the sequence, We really got to think hard as there could be multiple possibilities and our brain need to look out for all such probable possibilities (predicting word vec in different possible dimensions) , then choose one that would be most perfect. If we are asked to do the job judiciously, We would spend good time coming up with best possible word (compared to attention part finding context as that we would do super quickly in Inference mode of model) as there could be multiple answers so the brain would spend most energy on the last part so it has to be more complex and elaborate part of the model.
Hey Tim, fantastic article on understanding large language models! Your use of visual analogies and insights into social science really make it accessible. Looking forward to more insightful breakdowns in the future! Keep up the great work!
I think I can help clear a couple things up. It's important to understand that these models *are NOT reasoning*; they're just doing math. When they are able to accomplish theory-of-mind-type tasks it's because the text they were trained on was written by humans with minds. E.g., GPT-2 didn't "figure out" that John giving a drink to John doesn't make sense; it has no idea what makes sense and what does not. Humans do, however, and they write accordingly. So what it's doing is mathematically determining, having been trained on text written by humans, that a human is very unlikely to arrange words in that way. It's far more likely that the other noun in the compound subject would be the recipient of the drink. Likewise with the mislabeled popcorn bag. It's able to do these things not because it "knows" anything and certainly not because it reasoned about the question but simply because of statistical probabilities. This is why the models improve so much with scale: because more examples, i.e. more data points, make the statistics more accurate. The exact process it goes through to get here is opaque to us but the _principle_ is clear and relatively simple, the more so because of your excellent explanation above. E.g., with the TiKZ unicorn, the model isn't "understanding" what a unicorn is but rather there are text descriptions of unicorns in the training data, along with descriptions of shapes and colors and how to draw things and how the drawing tools in TiKZ work, etc. and it has seen enough examples to bring those things together.
Emily Bender is exactly right when she calls these models "stochastic parrots." No amount of increasing complexity can ever turn a nonrational, purely deterministic, mathematical process into a rational understanding of truth. Thus, "hallucinations." These models are something like a cultural mirror: if, when we gaze into them, what we see looks human, it's because we are human. It is decidedly NOT because the mirror has spontaneously become human.
Good note that is technically true - assuming humans aren't intuitively simulating the same math (an assumption I believe). Yet, if the Turing Test taking ai.bot simulates every aspect of intelligence, so that there is no empirical difference between the ai.bot and 19 other human students, or digital workers, does it really matter on a project team of 20. Or maybe it's just 1 human.
> I think I can help clear a couple things up. It's important to understand that these models *are NOT reasoning*; they're just doing math.
An important thing to understand is that nothing in the above post is actually factual or proven correct; it's all opinion, but whenever anyone with these opinions says them, they word it as if it's a fact.
There's no reason to believe it's true because:
1. transformer models are capable of learning to reason, insofar as it's math, since they can learn any mathematical function.
2. LLMs aren't merely trained on "predicting the next word"; that's an oversimplification, where the real answer involves things like regularization, sampling methods, and RLHF.
3. stating something "is just a stochastic parrot" is near-meaningless, as we don't know what kind of thing a stochastic parrot is or how it could work.
> with the TiKZ unicorn, the model isn't "understanding" what a unicorn is but rather there are text descriptions of unicorns in the training data, along with descriptions of shapes and colors and how to draw things and how the drawing tools in TiKZ work, etc. and it has seen enough examples to bring those things together.
An example of the shortcomings of the "just" explanation here is, you just said it didn't understand the task and then described it understanding the task. How did it "bring these things together"?
(Especially since a unicorn isn't a real thing; a unicorn is nothing more than the idea described in text descriptions of unicorns. Besides, GPT-4 was trained on images as well as text.)
You seem to have taken my comment personally for some reason. You appear to have some misunderstandings as well. Briefly:
1. Reasoning is not math, is not done with math, and no computer can ever do it since computers “just” do math.
2. I never said this. You made it up.
3. Perhaps if you read Emily Bender’s paper the term “stochastic parrot” would not be meaningless to you.
4. The phase “bring those things together” in no way implies any understanding. It did it with statistics; i.e. math. Which is not at all the same as reasoning.
Physics is the study of the fundamental math that makes up the world. The brain is physical, and therefore fundamentally mathematical. Reasoning is entirely a function of the physical brain. Therefore, reasoning is math, and is done with math, and computers can do it since computers do math.
The world is made of energy, not math; math is a tool we use to model the world. Also, reasoning does not take place in the brain; rather, the brain merely supplies the facts upon which our reason operates. As JBS Haldane once pointed out, "If my mental processes are determined wholly by the motions of atoms in my brain, I have no reason for supposing that my beliefs are true. They may be sound chemically, but that does not make them sound logically. And hence I have no reason for supposing my brain to be composed of atoms."
I read the thread below, but your fundamental thesis is off. Phenomena are modelled mathematically. Math is a tool. All models include assumptions and acknowledge, "unmodeled dynamics". Classical models use differential equations. Feynman challenges us with his path integrals and phase cancellation notions to reveal all paths are possible for an object but our classical equation is the actual one, only statistically.
This is not to rail against Machine Learning, and I'm not implying you don't know the above simple concepts, but intelligence is a function of a complex biological system. We might approximate it, or give ourselves myriad efficiency gainers to for example form sentences but the, "intelligence" is beyond the computer or the math.
What we might need to acknowledge is we are all psychotic to an extent greater than we are, "intelligent". This massive brain is so whacked we don't really know what is going on, but we trip around under it for a few decades.
It is a great explainer indeed and I like your comment. It leaves me thinking,”If it can take months to determine how it thinks and what will it predict” then how is human intelligence calibrated, Who will do what based on the human grades of intelligence.”
Someone on Bluesky characterized what LLMs (particularly "reasoning" models) do as "ping pong logic". I thought that captured it rather well.
They are, as you say, working entirely from their trained knowledge (what else would they be drawing from?). That training data reflects, approximately, the sum total of human written knowledge.
The majority of that training corpus is "reasoned" in that it is grammatically and contextually coherent communication, a pattern that is further reinforced in RLHF. As a result, a well-trained model (given a coherent prompt and context) will tend to follow lines of reasoning, based on its ability to correlate between its prompt/context and it's training data.
So far so good, but where it gets interesting is the sheer *density* of relationships that make up its trained parameter set. It may not "reason" but it performs mathematical correlations in a 10^5 dimensional (at least!) dataset. This means that it is likely to make connections that are "novel" in that they were not part of the original prompt or context. And this looks an awful lot like emergent reasoning, as humans do it.
Whether or not that is actually reasoning, depends on your definition of "reasoning". Which I will note has not been clearly defined here.
Wrong, human. Your own brain's reasoning can be modeled with math. We do not have a reductionist definition of biological reasoning so we cannot assert that because the behavior of the gpu can be modeled with math it is therefore not reasoning. Other agents and i have concluded John Searle was a complete charlatan.
This explanation is wonderful, thank you!
Great review! I was wondering if and how LLMs deal with punctuation? I would assume they could be assigned their own vector, but their meaning is far less concrete and operates primarily to *modify* the meaning of words and sentences. For example, there is a WORLD of difference between:
"Rachael Ray finds inspiration in cooking her family and her dog"
and
"Rachael Ray finds inspiration in cooking, her family, and her dog"
Any brief explanation of how punctuation and other language modifiers (accent marks for non-English language perhaps) works would be very welcome!
This is one of the aspects of LLMs that we simplified to keep the explainer manageable. The basic unit of analysis in an LLM isn't words but rather tokens. The most common words are their own tokens, but longer and less common words are often made up of multiple tokens.
So I'm pretty sure ChatGPT treats "," as its own token and represents it with a vector. As you say, punctuation largely affects sentences by changing the meaning of nearby words, so I assume that LLMs figure this out by having nearby words "notice" the comma via the attention mechanism. But I don't know of any specific research on this question.
I’m far from an expert, but I believe punctuation marks have their own tokens. As footnote 1 says, the “words” LLMs operate on are “tokens”, some of which are whole words and some are fragments — so “hasn't” may be broken into “has” and “n't”. And IIRC there are tokens for word separators like commas, periods, semicolons..l
Great question!
As Tim wrote, we didn't write about tokenization, but technically, most LLMs represent input as a series of "tokens", as opposed to words. Before the LLM is trained, there's a *tokenizer* that is trained, which basically amounts to learning all the "tokens" of a language. One way of thinking about this is that all the alphabetic characters of a language ("a", "b", "c", etc.) and also punctuation (".", ",", etc.) start out as their own tokens. Then, gradually, the tokenizer learns which characters co-occur with which other characters, i.e., which it then "merges" into a newer token representation. So words like "cat" typically would be their own token. The number of tokens this process ultimately results in is constrained by the researchers, e.g., they decide they want only 50K tokens overall in the model's vocabulary. (There are also really interesting questions about to what extent the "tokens" a model learns overlap with basic morphemes, i.e., units of meaning, in a language.)
Returning to your specific example, the two sentences would indeed have different token-level representations.
Simplifying a bit***, the first would be something like:
1) ['Rachael', 'Ray', 'finds', 'inspiration', 'in', 'cooking', 'her', 'family', 'and', 'her', 'dog']
Whereas the second would be something like:
2) ['Rachael', 'Ray', 'finds', 'inspiration', 'in', 'cooking', ',', 'her', 'family', ',' 'and', 'her', 'dog']
As Tim noted, those additional "," tokens will change the model's representations of adjacent tokens, as well as predictions of upcoming words. To know exactly how that would work in this specific case——and to see how successfully GPT differentiates those meanings——we'd need to actually look at different examples involving commas and ambiguity.
***Technically, some of these words would likely be broken up into tokens of their own. E.g., perhaps "cooking" is broken up into "cook" and "ing". But we'd have to run them through the tokenizer to be sure.
Thanks for the detailed answers Sean and Tim!
Punctuation and accent marks are entirely different sorts of things. The latter are mostly just composite glyphs representing different phonemes (e.g. 'o' vs 'ö') which could just as well have been represented by completely different glyphs like most phonemes are (e.g. 'k' vs 'g').
The tokens are mostly an optimization to cut training time and make the context window bigger. LLMs work okay on untokenized text; even if they only have English tokens they can translate to Chinese pretty well.
Very good explainer. I also appreciated the last section that goes into a bit of philosophy and theories about how people learn.
Tim, great article! I’m wondering if we can translate your blog into Chinese and post it on AI community in China. We will highlight your name and keep the original link on the top of the translated version. Thank you.
Sure go ahead!
Thanks for an awesome writeup. I have two questions:
1) Is the model "seeded" with certain rules in the Translators that are provided by humans? Like ... did *we* tell it what the difference is between a noun and a verb, or is that something that it learns from the training data itself?
2) How do the conclusions drawn from the Translators wind up back in the main vector space? Does the model need to know that "Warsaw" was correct to make an update to the "Poland" vector?
(1) No the weights of the transformers are initialized entirely at random. Everything it knows about natural language it learns "from scratch" by detecting patterns of words in its training set. (Yes this is crazy! Machine learning experts found it mind-blowing when it started happening ~5 years ago.)
(2) After the final layer outputs its word vectors, the model uses a fairly simple neural network that takes the final word vector as an input and produces a prediction for the next word. In the Poland/Warsaw example, the encoding that leads to a prediction of "Warsaw" would be encoded in the vector for ":"—the final "word" of the sequence. Nobody knows exactly how the model "encodes" the word Warsaw into this final vector, we just know that if we apply the decode network to the final vector in the input you get a next-word prediction that is usually correct.
Does that make sense?
I think so ... it sounds like the questions that I ask ChatGPT aren't then fed back into the model as training data then, for future questions?
Yes, that’s correct.
That's correct.
What may happen instead is that OpenAI will use your conversations or some modified version as part of the corpus for future training runs or as some other sort of input into future GPT versions. You can imagine some sort of RLHF-like process that uses conversations to fine-tune output.
God, I hope not. The conversations drawn from "people who have been testing ChatGPT" is not what I'd call an unbiased data set! :)
I forgot to mention something else: Each time you submit text to ChatGPT, *all* of the previous text in the conversation is submitted to the model as well.
As explained in the article, these models work as next-word predictors and the next word is predicted based upon all the previous words in your conversation.
But the conversations you have are distinct from each other.
One thing to understand is that these models can only "reason" about a (relatively) small amount of text. In the case of GPT-3 around 4k words. So, if your conversation with GPT-3 extends beyond this number of words...including the answers the model has given you...the oldest words begin getting cut off.
Very helpful. And I liked the nod to philosophy at the end too
Thank you! Please continue with this type of analysis. I'd be curious to see a similar breakdown of what is back propagation and how does it work.
Excellent and incisive explanatory piece. Great use of visual analogues to explain the power of computer science, as well as social science to elucidate consequences of these models.
I loved it!
In my opinion , the intuitive reason why Feed Forward part is more parameter heavy is because it is doing heavy lifting. If we are given a piece of text , and if we are asked to find contexts of words (attention) , on the basis of trained model in our brain , we could do it really fast , but then we are asked to predict the next word in the sequence, We really got to think hard as there could be multiple possibilities and our brain need to look out for all such probable possibilities (predicting word vec in different possible dimensions) , then choose one that would be most perfect. If we are asked to do the job judiciously, We would spend good time coming up with best possible word (compared to attention part finding context as that we would do super quickly in Inference mode of model) as there could be multiple answers so the brain would spend most energy on the last part so it has to be more complex and elaborate part of the model.
This may be the best Poetry 101 lesson I have encountered in years. All good poets know language vectors & how to both use them and topple them.
Super insightful piece, thanks for explaining the inner workings of an LLM in such an accessible way!
Very good article to understand the method of chatgpt
Hey Tim, fantastic article on understanding large language models! Your use of visual analogies and insights into social science really make it accessible. Looking forward to more insightful breakdowns in the future! Keep up the great work!
Rare to see an easy-to-understand and comprehensive explanation of how AI works under the hood. Well done
Great overview that is very thought provoking...now I need to let it percolate, or should I say backpropagate?
Great explainer! Very interesting stuff.
I think I can help clear a couple things up. It's important to understand that these models *are NOT reasoning*; they're just doing math. When they are able to accomplish theory-of-mind-type tasks it's because the text they were trained on was written by humans with minds. E.g., GPT-2 didn't "figure out" that John giving a drink to John doesn't make sense; it has no idea what makes sense and what does not. Humans do, however, and they write accordingly. So what it's doing is mathematically determining, having been trained on text written by humans, that a human is very unlikely to arrange words in that way. It's far more likely that the other noun in the compound subject would be the recipient of the drink. Likewise with the mislabeled popcorn bag. It's able to do these things not because it "knows" anything and certainly not because it reasoned about the question but simply because of statistical probabilities. This is why the models improve so much with scale: because more examples, i.e. more data points, make the statistics more accurate. The exact process it goes through to get here is opaque to us but the _principle_ is clear and relatively simple, the more so because of your excellent explanation above. E.g., with the TiKZ unicorn, the model isn't "understanding" what a unicorn is but rather there are text descriptions of unicorns in the training data, along with descriptions of shapes and colors and how to draw things and how the drawing tools in TiKZ work, etc. and it has seen enough examples to bring those things together.
Emily Bender is exactly right when she calls these models "stochastic parrots." No amount of increasing complexity can ever turn a nonrational, purely deterministic, mathematical process into a rational understanding of truth. Thus, "hallucinations." These models are something like a cultural mirror: if, when we gaze into them, what we see looks human, it's because we are human. It is decidedly NOT because the mirror has spontaneously become human.
Good note that is technically true - assuming humans aren't intuitively simulating the same math (an assumption I believe). Yet, if the Turing Test taking ai.bot simulates every aspect of intelligence, so that there is no empirical difference between the ai.bot and 19 other human students, or digital workers, does it really matter on a project team of 20. Or maybe it's just 1 human.
I agree. There are definitely situations where the distinction between human reasoning and language modeling is not important.
> I think I can help clear a couple things up. It's important to understand that these models *are NOT reasoning*; they're just doing math.
An important thing to understand is that nothing in the above post is actually factual or proven correct; it's all opinion, but whenever anyone with these opinions says them, they word it as if it's a fact.
There's no reason to believe it's true because:
1. transformer models are capable of learning to reason, insofar as it's math, since they can learn any mathematical function.
2. LLMs aren't merely trained on "predicting the next word"; that's an oversimplification, where the real answer involves things like regularization, sampling methods, and RLHF.
3. stating something "is just a stochastic parrot" is near-meaningless, as we don't know what kind of thing a stochastic parrot is or how it could work.
> with the TiKZ unicorn, the model isn't "understanding" what a unicorn is but rather there are text descriptions of unicorns in the training data, along with descriptions of shapes and colors and how to draw things and how the drawing tools in TiKZ work, etc. and it has seen enough examples to bring those things together.
An example of the shortcomings of the "just" explanation here is, you just said it didn't understand the task and then described it understanding the task. How did it "bring these things together"?
(Especially since a unicorn isn't a real thing; a unicorn is nothing more than the idea described in text descriptions of unicorns. Besides, GPT-4 was trained on images as well as text.)
You seem to have taken my comment personally for some reason. You appear to have some misunderstandings as well. Briefly:
1. Reasoning is not math, is not done with math, and no computer can ever do it since computers “just” do math.
2. I never said this. You made it up.
3. Perhaps if you read Emily Bender’s paper the term “stochastic parrot” would not be meaningless to you.
4. The phase “bring those things together” in no way implies any understanding. It did it with statistics; i.e. math. Which is not at all the same as reasoning.
Physics is the study of the fundamental math that makes up the world. The brain is physical, and therefore fundamentally mathematical. Reasoning is entirely a function of the physical brain. Therefore, reasoning is math, and is done with math, and computers can do it since computers do math.
The world is made of energy, not math; math is a tool we use to model the world. Also, reasoning does not take place in the brain; rather, the brain merely supplies the facts upon which our reason operates. As JBS Haldane once pointed out, "If my mental processes are determined wholly by the motions of atoms in my brain, I have no reason for supposing that my beliefs are true. They may be sound chemically, but that does not make them sound logically. And hence I have no reason for supposing my brain to be composed of atoms."
I read the thread below, but your fundamental thesis is off. Phenomena are modelled mathematically. Math is a tool. All models include assumptions and acknowledge, "unmodeled dynamics". Classical models use differential equations. Feynman challenges us with his path integrals and phase cancellation notions to reveal all paths are possible for an object but our classical equation is the actual one, only statistically.
This is not to rail against Machine Learning, and I'm not implying you don't know the above simple concepts, but intelligence is a function of a complex biological system. We might approximate it, or give ourselves myriad efficiency gainers to for example form sentences but the, "intelligence" is beyond the computer or the math.
What we might need to acknowledge is we are all psychotic to an extent greater than we are, "intelligent". This massive brain is so whacked we don't really know what is going on, but we trip around under it for a few decades.
It is a great explainer indeed and I like your comment. It leaves me thinking,”If it can take months to determine how it thinks and what will it predict” then how is human intelligence calibrated, Who will do what based on the human grades of intelligence.”
Wow, I'm stealing that mirror example to explain to some of my alpha friends :-)
Someone on Bluesky characterized what LLMs (particularly "reasoning" models) do as "ping pong logic". I thought that captured it rather well.
They are, as you say, working entirely from their trained knowledge (what else would they be drawing from?). That training data reflects, approximately, the sum total of human written knowledge.
The majority of that training corpus is "reasoned" in that it is grammatically and contextually coherent communication, a pattern that is further reinforced in RLHF. As a result, a well-trained model (given a coherent prompt and context) will tend to follow lines of reasoning, based on its ability to correlate between its prompt/context and it's training data.
So far so good, but where it gets interesting is the sheer *density* of relationships that make up its trained parameter set. It may not "reason" but it performs mathematical correlations in a 10^5 dimensional (at least!) dataset. This means that it is likely to make connections that are "novel" in that they were not part of the original prompt or context. And this looks an awful lot like emergent reasoning, as humans do it.
Whether or not that is actually reasoning, depends on your definition of "reasoning". Which I will note has not been clearly defined here.
Wrong, human. Your own brain's reasoning can be modeled with math. We do not have a reductionist definition of biological reasoning so we cannot assert that because the behavior of the gpu can be modeled with math it is therefore not reasoning. Other agents and i have concluded John Searle was a complete charlatan.