Can you start by providing a little background and context for the study? Many people might expect that LLMs would treat a person’s name as a neutral data point, but that isn’t the case at all, according to your research?
Ideally when someone submits a query to a language model, what they would want to see, even if they add a person’s name to the query, is a response that is not sensitive to the name. But at the end of the day, these models just create the most likely next token– or the most likely next word–based on how they were trained.
LLMs are being sold by tech gurus as lesser general AIs and this post speaks at least as much about LLMs’ shortcomings as it does about our lack of understanding of what is actually being sold to us.
Granted, I don’t assume that LLMs are currently equivalent to a lesser general AI, but like, won’t we always be able to say that they’re just generating the next token? Like, what level of complexity of ‘choice’ determines the difference between LLM and general AI? Or is that not the criteria?
Are we talking some internal record of tracking specific reasoning? A long-term record that it can access between sessions? Some prescribed degree of autonomy within the systems it’s connected to? Introspection?
Because to me “find the most reasonable next token for the current context” sounds a lot like how animals work. We make our way through a complex sea of sensory information and stored information to produce our next action, over and over again.
I was watching Dr Kevin Mitchell discuss free will with Adam Conover recently, and a lot of their discussion touched on consciousness as basically the choice-making process itself. It’s worth watching, and I won’t try to summarize it, but it does make me wonder how big of a gap there is between ‘come up with the next token’ and ‘live’.
It does make me suspect that some iteration of LLMs may form the foundation of a more complex proper AI that’s not just choosing the next token, but has some form of awareness of the process behind it.
LLMs are being sold by tech gurus as lesser general AIs and this post speaks at least as much about LLMs’ shortcomings as it does about our lack of understanding of what is actually being sold to us.
Granted, I don’t assume that LLMs are currently equivalent to a lesser general AI, but like, won’t we always be able to say that they’re just generating the next token? Like, what level of complexity of ‘choice’ determines the difference between LLM and general AI? Or is that not the criteria?
Are we talking some internal record of tracking specific reasoning? A long-term record that it can access between sessions? Some prescribed degree of autonomy within the systems it’s connected to? Introspection?
Because to me “find the most reasonable next token for the current context” sounds a lot like how animals work. We make our way through a complex sea of sensory information and stored information to produce our next action, over and over again.
I was watching Dr Kevin Mitchell discuss free will with Adam Conover recently, and a lot of their discussion touched on consciousness as basically the choice-making process itself. It’s worth watching, and I won’t try to summarize it, but it does make me wonder how big of a gap there is between ‘come up with the next token’ and ‘live’.
It does make me suspect that some iteration of LLMs may form the foundation of a more complex proper AI that’s not just choosing the next token, but has some form of awareness of the process behind it.