by Corrie Pikul-Browm
February 15, 2023
Extract:
Read more here: https://www.futurity.org/author/corrie-pikul/(Futurity) the backbone of ChatGPT is a state-of-the-art kind of artificial neural network called a transformer network. These networks, which came out of the study of natural language processing, have recently come to dominate the entire field of artificial intelligence. Transformer networks have a particular mechanism that computer scientists call “self-attention,” which is related to the attentional mechanisms that are known to take place in the human brain.
Another similarity to the human brain is a key aspect of what has enabled the technology to become so advanced, (Thomas) Serre (professor of cognitive, linguistic, and psychological sciences and of computer science) said. In the past, he explained, training a computer’s artificial neural networks to learn and use language or perform image recognition would require scientists to perform tedious, time-consuming manual tasks like building databases and labeling categories of objects.
Modern large language models, such as the ones used in ChatGPT, are trained without the need for this explicit human supervision. And that seems to be related to what Serre referred to as an influential brain theory known as the predictive coding theory. This is the assumption that when a human hears someone speak, the brain is constantly making predictions and developing expectations about what will be said next.
…
On a darker note, in the same way that the human learning process is susceptible to bias or corruption, so are artificial intelligence models. These systems learn by statistical association, Serre said. Whatever is dominant in the data set will take over and push out other information.
…
NOPE, CHATGPT DOESN’T DREAM
One area in which human brains and neural networks diverge is in sleep—specifically, while dreaming. Despite AI-generated text or images that seem surreal, abstract, or nonsensical, (Elle) Pavlick (assistant professor of computer science and a research scientist at Google AI) said there’s no evidence to support the notion of functional parallels between the biological dreaming process and the computational process of generative AI. She said that it’s important to understand that applications like ChatGPT are steady-state systems—in other words, they aren’t evolving and changing online, in real-time, even though they may be constantly refined offline.
For more discussion on how artificial intelligence models may be susceptible to bias or corruption: https://www.futurity.org/stroke-risk-a ... 863922-2/