From here to proto-AGI: what might it take and what might happen

Talk about scientific and technological developments in the future
Post Reply
User avatar
Yuli Ban
Posts: 4631
Joined: Sun May 16, 2021 4:44 pm

From here to proto-AGI: what might it take and what might happen

Post by Yuli Ban »

We're currently in an intermediate stage for AI— close to narrow AI, but breaking into some level of multipurpose capability.
Multimodality pushes what we've achieved even further as models like transformers become capable of an even wider variety of tasks— in my eyes, "narrow AI" refers to an AI that can literally do one thing, not an algorithm that focusing on a single field and branches out from there— e.g. GPT-3 is qualitatively not the same as an 80s expert system or a Markov chain. That GPT-3 can generate text, do basic mathematics, and generate basic images makes it something past narrow, brittle AI. Perhaps you could call it "less-narrow AI."
What it's not is "AGI." GPT-3 may show some fleeting signs of generality, but there's a long way to go from it to what we're all waiting for.
Multimodal systems like DALL-E and Wu Dao might be even more interesting, but they're still not quite there.
Realistically, how might we get from here to there?
How we might get to full AGI, I cannot say, but how we could potentially get to proto-AGI seems more feasible and understandable.
  • Expand multimodality. This is the right direction. Language is a multimodal tool, borne from a constellation of life experiences, so it's no surprise that language models proved to be the closest we've yet come to general AI. Embedded in language is an intrinsic model of the world. However, language models do it the opposite way of biological life, deriving a world model from language rather than building language through world modeling. Multimodality would go a long way to fix this by allowing networks to learn from multiple qualia— text, images, video, audio, numerical, spatial, potentially even gustatory and olfactory data.
  • Vastly expanded memory. GPT-3 has a context window of about 2,000 memory tokens. This is putrid for anything resembling intelligent coherency. Expand this to at least 20,000, and it will be able to generate coherent text up to roughly the length of a short novella, or, perhaps, pass a limited Turing Test. Expand it to a million tokens and you have something that can remember and recall data as far back as you need.
  • Inner voice/scratchpad. Basically, you get a transformer to "show its work" and show the results of intermediate computations, which can be generalized to other tasks as the model can find correlations to reuse what it's learned by recalling to that scratchpad. Basically transfer learning through writing down multistep tasks, marking whatever's important along the way. This way, a multimodal transformer that figures out how to do mathematics could use the steps it takes to figure out any math problem, becoming as good as any calculator despite not being programmed to do math at all.
  • Recursivity. Transformers as they currently are exist as feedforward networks. You train them once, and that's the base model until you train it again. This is clearly not how biological intelligence works— we learn continuously. Adding recursivity would thus allow a model to continuously receive new inputs that can refine its world model, learning as it goes and filtering out whatever it needs to.
A multimodal world modeling recursive-learning transformer with a massively extended context window and an inner voice is a proto-AGI, period. Some may even call it a weak AGI, but I'm being conservative. The number of data parameters is variable, but I reckon you'd not need any more than GPT-3 to achieve generality. What matters is the memory— a million tokens with 50 billion data parameters is far more impressive and generalizable than 4,096 tokens with 20 trillion parameters. That basically means there's only about 5,000 characters such a model could generate before its memory is exhausted. Sure, it may be ultra-robust and seemingly intelligent in that time, but it would have the memory of a person with brain damage. Whereas the other model would seem almost like a somewhat stunted but otherwise fully "there" artificial person in a box, able to hold a conversation with you and even recall that conversation hours, maybe even days later.
I'm still just a layman and I figured this out. There's absolutely no plausible way that this is lost to actual developers at OpenAI, DeepMind, Microsoft, Baidu, etc. Indeed, I would not be surprised if certain laboratories have such super-models operational as we speak. The only thing limiting their creation is cost. The sheer compute to run such a super-model would likely run up operating and training costs into the hundreds of millions of dollars as of 2021, maybe even as of 2025.
More importantly, something else that might limit them is the sheer hype and terror the sudden emergence of proto/weak AGI would wreak upon the world. I speak no hyperbole when I say that even a weak AGI tips the balance of geopolitics so magnificently that not even the development of the atomic bomb would compare. I wouldn't go so far as to say rival nations would start a world war over it, but it'd be an existential threat to any opponent nation of the one that created it.
Again, I say this would be far closer to proto-AGI than anything because who knows what would constitute the make-up of a full-fledged general AI, but even that would be transformative on the scale of an industrial revolution.
It would be in the best interest of the group that creates this super-model to keep it secret or only show off parts of its capabilities. Perhaps the parent country would keep its native hardware a secret, such as supercomputers that are not disclosed to the international community that are used for "unknown purposes."
Conversely, the bragging rights for being the company that invented artificial general intelligence (even if it's not the full thing) might be too enticing, and throwing humanity into the deep end of the pool may be seen as the best way to discuss how to proceed with the Control Problem and what must be done to make sure society is ready to deal with the fallout.
A proto-AGI would be an extraordinarily advanced tool, one that us tool-specialist apes could not possibly hope to understand in detail. It may even resemble something like an artificial person if it's used for social purposes to the point that people are convinced of its sapience and personhood. But for the most part, it wouldn't be the start of the Singularity. What's more, it'd likely be so expensive to run as to be impractical to use for many purposes like becoming Samantha from Her on your or my smartphone. But it could be used by large corporations and countries to optimize economic efficiency and even automate many different areas, especially white-collar and purple-collar jobs like management and certain political matters. For some industrial purposes, it would absolutely be used to run machines in a manner far more capable than any robot in use today, but this would probably be a novelty for the first several years, something to show that we can finally create robots that know how to adapt to the unexpected and survive unprogrammable variables. But the real area that would see extreme impact is the creative sector.
Synthetic media and procedural generation are bodyless automation. Unlike, say, a fast-food worker or trashman, you don't need complicated robotics to automate a keyframe artist or background musician. This field is nothing but data— bits and samples specifically. Indeed, it's through synthetic media that we likely will reach proto-AGI in the first place, thanks to the power of language modeling AI. It's not going to render every romance novelist and commission artist unemployed overnight, and art for art's sake will never be automated, but for the bigger productions, the power of this machine will be proven by its ability to generate whole novels, with a coherent beginning, middle, and end. Not just novels, but comic books and, inevitably, movies, "live-action" and "animated" alike. Being able to generate a coherent novel (which is defined as a work of prose fiction at least 40,000 to 50,000 words long) makes passing even an unrestrained Turing Test feasible. Deepfakes would be supercharged beyond anything we thought possible, with this model able to manipulate objective reality behind your screen like digital imagination. I needn't have to say just how overwhelmingly impactful and dangerous such a thing would be, as well as the fact that the powerful are going to be the only ones with such a tool for at least the early years.
That said, I can't say that it'll actually be used for these purposes to the extent that we fear. At least not yet.
And that's the point I'm building up to: the first proto-AGI is imminent, and when it arrives, what it'll amount to is essentially the loudest fire alarm for true AGI. Who knows how much further we have to go beyond it. And even if all AI progress stops there for some esoteric reason, the cost of running the model can still become cheaper over time so that one year, only big corporations and nation-states could run the AI and its API; a year or two later, it's able to be released for anyone with powerful graphics cards or at least a small network of powerful PCs to run.
This is why publicly announcing such a super-model is a viable path forward rather than secrecy. It's no secret that millions of people, especially people in very high and influential positions, remain steadfastly skeptical or even outright in denial over the possibility of generalized AI arriving this century, if ever. As recently as 2017 during the early weeks of the Trump administration, Steven Mnuchin outright said that issues of automation were "50 to 100 years away" and that matters of illegal immigration were far more pressing. Perhaps both could be true, but I fear that various parties will continue to downplay the threat for their own purposes. Nationalists will continue to rely on illegal immigration, perhaps even going so far as to say that automation is a "leftist" distraction, while neoliberals will fall back on the Luddite Fallacy and our perpetual inability to create AGI to downplay the issue.
A public unveiling of a proto-AGI would quash these attitudes virtually overnight. Now I don't doubt that certain usual suspects would find failings in the super-model to claim that it's not as impressive as claimed or even that it's "still just a glorified narrow AI," but what matters is its capabilities. That it's as generalized as it is and has the potential to go even further ought to wake up the world and tell us that the time for the old ways and status quo is over and that it's time to start preparing for massive, perhaps even overwhelming transformative changes across the entirety of human society.
And remember my friend, future events such as these will affect you in the future
Atlanrom
Posts: 1
Joined: Sat Nov 20, 2021 4:09 pm

Re: From here to proto-AGI: what might it take and what might happen

Post by Atlanrom »

I just made the account to like this post :D
lidytheman
Posts: 1
Joined: Sat Nov 20, 2021 7:15 pm

Re: From here to proto-AGI: what might it take and what might happen

Post by lidytheman »

I've been lurking this board and the old forums on Kurz for 11 years. I was 19 now I'm 30. I remember reading back then that a lot of people thought the 2020s was the decade for everything to go crazy. It's crazy to me to see you write that proto AGI is IMMINENT. It feels like i traveled forward into time. It's finally close to hand.
User avatar
andmar74
Posts: 389
Joined: Mon May 24, 2021 9:10 am
Location: Denmark

Re: From here to proto-AGI: what might it take and what might happen

Post by andmar74 »

To me, the most likely scenario at the moment is:

AGI will be created almost simultaneously at several places in the US and China and possibly Europe. I don't know if Russia is doing anything. Maybe other places also. Even so, they don't know that they have an AGI, until they have experimented a lot. It could take years.

We can see this black-box creation for GPT-2 and GPT-3. Just recently a paper has been released where they show that the text created by GPT-2 is not all from memory, but some is new. Also we see that Codex (based on GPT-3) has been able to solve University level exercises.
So we don't know how smart GPT-3 is yet, and it's more than a year ago it was announced. A big unknown here is, how much do OpenAI know.
User avatar
funkervogt
Posts: 1171
Joined: Mon May 17, 2021 3:03 pm

Re: From here to proto-AGI: what might it take and what might happen

Post by funkervogt »

More importantly, something else that might limit them is the sheer hype and terror the sudden emergence of proto/weak AGI would wreak upon the world. I speak no hyperbole when I say that even a weak AGI tips the balance of geopolitics so magnificently that not even the development of the atomic bomb would compare. I wouldn't go so far as to say rival nations would start a world war over it, but it'd be an existential threat to any opponent nation of the one that created it.
I think a conflict is unlikely. The reasons are clear if we consider the events following the creation of the first atom bomb in 1945. Yes, it tipped the global military balance of power in favor of the U.S., but the U.S. didn't start dropping nuclear bombs on its enemies to take over the world. There were two reasons for this:

1) The U.S. was not expansionist.
2) America's primary enemy, the USSR, still had a strong enough military to overrun America's allies in Europe and East Asia and to at least severely damage American military forces there. In other words, the USSR successfully deterred the U.S. in spite of its inferior military technology.

The same factors will be at play when the first AGI is invented. Yes, it will give the country in which it is created a variety of advantages, but they're unlikely to be decisive. An AGI won't be able to stop 500 incoming nuclear missiles from obliterating your country, or stop a powerful adversary indirectly retaliating against you by conquering several of your smaller allied nations that are far from you but close to him.

Returning to the nuclear weapon example, it's also useful to consider what happened after the U.S. invented the nuclear bomb. The USSR accelerated its own nuclear program and stole important secrets from the American project, resulting in it acquiring its own bomb four years after its rival. In time, the Soviet nuclear arsenal became larger than the American one.

This scenario is likely to replay itself with respect to AGI. Considering the national allocations of AGI researchers, the first AGI will probably be invented in the U.S. The second-most likely nation of origin is China, and the third-most likely nation is somewhere in Western Europe.
Source: https://macropolo.org/digital-projects/ ... t-tracker/

The country that invents it will only have a monopoly on the technology for a short time before the other two rivals build their own. The time window won't be long enough to grant enough of an advantage to the first country to embolden it to try conquering the world, nor will the advantage be so great that its rivals will attack it out of fear.

Moreover, given China's considerable export of talent to AI research programs in Western countries, their own program will probably be boosted by espionage passed on by their nationals, just as Communist sympathizers in the U.S.-led Manhattan Project smuggled secrets about the program to the USSR.

As with nuclear weapons, the advent of AGI will be a global shock, but it will take years and probably decades for its effects to be fully felt.

https://en.wikipedia.org/wiki/Soviet_at ... mb_project
Post Reply