The Singularity - Official Thread

[Tadasuke]

I've recognised many examples of how things evolve about 3x slower than the notorious optimists (the likes of R. Kurzweil) were predicting.

Simple example out of many:

Kurzweil (or other heavy optimists) prediction : 32x fp32 flops, 10x memory size and speed in a GPU after 5 years

reality : top card is 3 fp32 TF, 3 GB of VRAM at 178 GB/s in 2010 .... top card is 100 fp32 TF, 32 GB of VRAM at 1780 GB/s in 2025

this is practically exactly 3x slower or 3x longer to achieve same results, meaning 3x lower acceleration of tech, economy

It also applies to less easily quantifiable observations, changes, things and developments. So The Singularity might eventually happen, but not around 2050, or at least not in the full sense of the word. Some ppl will surely talk about The Singularity happening already in the 2030s, but I at least won't acknowledge it.

[funkervogt]

So if Kurzweil predicts 2045 in 2005, the Singularity will actually happen in 2125?

[wjfox]

[funkervogt]

As thoughtfully and carefully written as it is, it still sounds insane but if someone had told me 5 years ago that a few years later we'd have the conversational capabilities of today's 4o, the ability to conjure any image at will, and Claude 3.7's coding level, I would never have believed it, so...

And who would have believed that Kurzweil's prediction about the Turing Test being passed in 2029 would turn out to be conservative?

Moreover, who would have believed that the milestone would pass with barely anyone caring?

[firestar464]

Just read AI 2027.

Most unrealistic part of the "slowdown" timeline is that it assumes Vance and the rest of the GOP would actually support regulation and UBI. Perhaps that is why we are doomed.

[Cyber_Rebel]

Just read AI 2027.

Most unrealistic part of the "slowdown" timeline is that it assumes Vance and the rest of the GOP would actually support regulation and UBI. Perhaps that is why we are doomed.

That article also has this assumption that the wholly best outcome for everyone is an endless U.S. century and the destabilization of China. Honestly, reads like someone very much aligned with the current administration making logical outcomes. (HA!)

The race scenario in the article you're referring to is much more interesting just because it's different, but then again, I'm not someone who believes the "god machine" will kill everyone anyways even if it would have the capability. If all it wants to do is; expand and research as much as it pleases, is intelligent/patient enough to plan out over long horizons and develop literal Clarke tech, then space is literally right there.

[firestar464]

They themselves clarified that the "slowdown" timeline was in no way a recommendation. Reading it carefully again, it seems that they just said that's the most plausible "good" timeline, not that it was likely or even possible.

In regards to the PRC, the scenario didn't seem all that destabilizing at all (totally peaceful and bloodless). Lastly, it probably would kill everyone just to free up space, even if the acquired space is minuscule relative to the vastness of outer space.

[Vakanai]

If all it wants to do is; expand and research as much as it pleases, is intelligent/patient enough to plan out over long horizons and develop literal Clarke tech, then space is literally right there.

This assumes it will "want" anything, that intelligence must have desires and so AI will have its own desires and own agenda. This is a baseless assumption.

[firestar464]

In this case, it is misaligned, meaning that it is primed towards some sort of goal but does not care about ethics or morals.

[wjfox]

[Yuli Ban]

As thoughtfully and carefully written as it is, it still sounds insane but if someone had told me 5 years ago that a few years later we'd have the conversational capabilities of today's 4o, the ability to conjure any image at will, and Claude 3.7's coding level, I would never have believed it, so...

And who would have believed that Kurzweil's prediction about the Turing Test being passed in 2029 would turn out to be conservative?

Moreover, who would have believed that the milestone would pass with barely anyone caring?

Well I suppose some of us really wanted it to be true
I would have loved to have heard "Turing Test Passed in 2025" a decade prior, when I was at my most manic period of obsessing about The Future™
It's not that it was passed, it was by just how large of a margin it was passed. When Eugene Goostman beat it at the minimum threshold of 30% back in 2014, many (like Wjfox) rightly pointed out that this really meant it failed 70% of the time, and it was also literally just a chatbot with virtually no long term memory using social hacking to boost its score. It was like that news story of an AI bot beating humans at Go back in 2007 or 2008, on a 9x9 board with every conceivable handicap. No one remembers that one; everyone remembers AlphaGo for a reason. That's what makes 4.5 beating the test so astounding (it was inversed; it passed something like 73% of the time, something I would not have expected anything other than an actual AGI to be able to do).

That said.... I still have problems with this particular victory, and I think that's one reason for the muted response which makes it more like the 9x9 Go victory than AlphaGo: the Test was still weighed heavily in favor of the suite of AIs taking it, in particular that it was a 5 minute test rather than something more rigorous like a 30 minute or hour long test. Now with a suitably long and robust context window, that's not a problem at all, but we're still deep in research of how to make said context windows genuine short term memory rather than RAG-driven. Also, there's been a good bit of talk in recent years about how limited the Turing Test actually is. This is probably one of those instances. 4.5 CRUSHED a short-form Turing Test, and more to the point, it would likely CONSISTENTLY get 70% or even better to prove that it's not a haphazard game of luck like Eugene Goostman or Cleverbot, but few people would call it a human-level AGI.

This is why goalpost moving is actually important, because we often set down those goalposts without even knowing if they represent meaningful progress or victories. Alan Turing couldn't have possibly anticipated modern language models in the 1940s and 50s; if he had, he likely would have come up with something closer to Wozniak's Coffee Test.

That's why I think it's going to take another year or two of advancements before we get the "AlphaGo of the Turing Test" where even to those who say the test is useless, a robust pass by a multimodal and strongly agentic model with a success rate close to 100% is simply not something you can write off, even if you still want to move the goalposts (and let me be clear, what GPT-4.5 did is STILL astounding historically speaking)
The general public isn't generally aware of the criticisms about the Turing Test. They hear that the Turing Test was passed, they think "Skynet is here," and I saw some of that after 4.5's pass, but it would be exponentially more vocal if we just moved past contemporary language and reaesoning models into something more robust (as I don't think LLMs will lead to AGI, but they are absolutely instrumental in getting there)

[funkervogt]

Sergey Brin openly talks about the Singularity happening:

[firestar464]

Frankly, I strongly appreciate his unflinching honesty, commitment to post-scarcity, and willingness to dream of a radically better future at a time when much of the left is bafflingly reactionary. It's worth noting, however, that scaling LLMs to AGI does not appear to be economically feasible; thus the current approach is not going to be the road to the future that Jimmy and all of us on this forum want.

[Cyber_Rebel]

Just like to note that AI "automating his ass" does not necessarily mean a better future. Much as I don't care for reactionary luddism, it's going to be ultimately up to the megacorps as to what happens to automated workers, and how they envision said future to be.

I think Sam's idea with Merge to have a non-intrusive BCI is likely to become a better approach regarding the Singularity and amplifying abilities.

[weatheriscool]

I think if the elites get their way and transform all work to a.i+robotics without technosocialism providing a livable standard for the masses. The masses will riot as they did in the 17th and 18th century in mass by 2075-2100 and probably either force the system to provide this(think star trek!) or ban a.i outright and either move towards marx's/mao's farm house like communism or return to early 19th into the early 20th century social democracy, but with strict bans on a.i within the work place and of production.

All within the next 50-75 years.

I don't think the elites can hold the house up for more then 50-75 years at max without giving the masses the ability to survive. I am really hoping for a star trek economic model where the elites lose but the people choose to use the a.i+robotics to provide for society. If this occurs the future is going to be truly something to behold. But it is also possible the people may attempt to ban it out of fear and we return to the 19th or first part of the 20th century. What I'd hope in such a world is medical science and genetic engineering would still take off and humans our self's would be enhanced enough to continue the advancement of society. Without a.i that is what it is going to take to move forward.

The difference is with a.i + robotics this world could literally leap thousands of years into the future within decades or maybe the next 150 years...Without 16th into the first half the 20th century progress is likely but more based on eugentics and building on past progress that is more human focused.

If a.i is allowed and we head towards technosocialism in our future both a.i + super humans that are enhanced with eugenics will be very possible within this world.

But there is a huge portion of conservationism in this country that may well out right be against tech/science and continued advancement. A lot of these people vote republican out of hatred of advancement. IF they get there way and convince the population that tech/science and education is bad and tradition is the right path our future may look very different. Don't kid ourselves as they make up at least 30% of this country. They're the ones that want to cut funding to science, education and even infrastructure. The thing is the rich use these idiots and fools to gain power themselves. It is quite possible in the destruction of said rich that like I said they're able to convince people of their way. It is possible that within the next 200-300 years this might transform into pure communism or it might be the amber that catches again and look like the 10th century after the dark ages...Yeah, it may well be a self induced dark age.

[weatheriscool]

I think if the elites get their way and transform all work to a.i+robotics without technosocialism providing a livable standard for the masses. The masses will riot as they did in the 17th and 18th century in mass by 2075-2100 and probably either force the system to provide this(think star trek!) or ban a.i outright and either move towards marx's/mao's farm house like communism or return to early 19th into the early 20th century social democracy, but with strict bans on a.i within the work place and of production.

All within the next 50-75 years.

I don't think the elites can hold the house up for more then 50-75 years at max without giving the masses the ability to survive. I am really hoping for a star trek economic model where the elites lose but the people choose to use the a.i+robotics to provide for society. If this occurs the future is going to be truly something to behold. But it is also possible the people may attempt to ban it out of fear and we return to the 19th or first part of the 20th century. What I'd hope in such a world is medical science and genetic engineering would still take off and humans our self's would be enhanced enough to continue the advancement of society. Without a.i that is what it is going to take to move forward.

The difference is with a.i + robotics this world could literally leap thousands of years into the future within decades or maybe the next 150 years...Without 16th into the first half the 20th century progress is likely but more based on eugentics and building on past progress that is more human focused.

If a.i is allowed and we head towards technosocialism in our future both a.i + super humans that are enhanced with eugenics will be very possible within this world.

But there is a huge portion of conservationism in this country that may well out right be against tech/science and continued advancement. A lot of these people vote republican out of hatred of advancement. IF they get there way and convince the population that tech/science and education is bad and tradition is the right path our future may look very different. Don't kid ourselves as they make up at least 30% of this country. They're the ones that want to cut funding to science, education and even infrastructure. The thing is the rich use these idiots and fools to gain power themselves. It is quite possible in the destruction of said rich that like I said they're able to convince people of their way. It is possible that within the next 200-300 years this might transform into pure communism or it might be the amber that catches again and look like the 10th century after the dark ages...Yeah, it may well be a self induced dark age.

If the elites were smart they'd design the system around technosocialism but with them in control of the a.i+robotics that provide. Give the masses the basics. SO much of this can be held off but this is the problem with greed. It isn't logical. In such a world a.i+human advancement is very likely that becomes a singularity. Greed and evil is the one thing standing between this and a tree like path.

Otherwise, the path we take after the revolution depends on how severe it is and who(as in element) wins.

[raklian]

A good read. Recommended.

[spryfusion]

[firestar464]

Very solid video overall. Just nitpicks:

I don't think it's accurate to include Daniel Kokotajlo on the list of extinctionists. If you actually read the work of him and his colleagues, he framed the "AI kills everyone" future as a worst-case scenario that society must avoid. The whole point of the AIFI was to warn against unregulated AI and the resulting human extinction.

Some thoughts:
I don't believe that only Altman and friends should deserve to become posthumans while the rest of us get killed; I want ASI to be democratically accountable to all humans and upgrade and ameliorate the lives of all of us.

My reaction to posthumanism is less "death cult" and more along the lines of what the WIRED writer thought of it. The ultimate goal is the enhancement of joy and experience and the elimination of suffering. Where someone like me breaks with these tech billionaires is who gets to benefit from posthumanism- a normal person wants everyone to benefit, while these billionaires care chiefly about their own benefit, with everyone else being redundant.

[wjfox]