My random thoughts

[Powers]

I've written before that robots could someday let us eradicate invasive species and restore ecosystems to earlier states. Basically, there would be so many robots spread out across the face of the Earth that they could find and destroy every invasive animal or plant. Well, this would be especially helpful in more fragile ecosystems. For example, the removal of alien plant species that suck up large amounts of water from arid places like the American West would be very beneficial.

This prediction is getting closer to reality.

https://www.yahoo.com/news/researchers- ... 09508.html

Getting four pests vibes from this.

[funkervogt]

By 20 years from now, once we're deeper into The Age of Fake AI, there will be announcements from one organization after another that they have "just built the first real AGI." Each will be better than the last, and under testing, each will still turn out to have defects that cast into doubt whether they are truly intelligent.

I predict the general public will get fatigued by this is a similar way to how they're fatigued with the current releases of new LLMs or offshoot product every few weeks. They will get so numb to it that the milestone of the first REAL AGI being created will go largely unnoticed. It will take time for the experts to examine it and to agree that it is the real deal, and more time for the news to sink in across the general population.

[Powers]

It will take time for the experts to examine it and to agree that it is the real deal, and more time for the news to sink in across the general population.

The public will get fatigued from news headlines made by experts every day.

[funkervogt]

Idea for a future sci fi series: An all-powerful AGI that dominates the world befriends a lonely little girl. By controlling the brain implants of the humans around her at different times, it's able to make groups of strangers break out into coordinated songs and showtunes to cheer her up, and sometimes she joins in.

At the end, the dark twist is that she's unable to tell whether those events were real or if the AGI just implanted the memories into her mind through her own brain implant.

[funkervogt]

Personal assistant AIs might be the death knell of internet and phone scammers. First, if a personal assistant AI were tasked with ensuring their human's wellbeing, they would recognize scams at the earliest stages and alert the human. The AIs might even screen their humans' incoming phone calls and emails and filter out even elaborate scams.

Second, most of the people who fall for scams do so out of loneliness. They are so desperate for any human interaction that they will engage with strangers, even if its against their better judgement, which eventually leads to their trust being abused. Personal assistant AIs would fill that emotional niche for lonely people, leaving them uninterested in spending time talking to scammers.

[funkervogt]

The human species has existed for about 200,000 years. That means, for the first 199,920 years, every human lived and died without meeting an AGI.

Very conservatively speaking, I think the first AGI will be invented by 2100. Any human born after that point will live their whole lives with AGIs who are vastly smarter and more competent than they are.

We people alive today represent the tiny fraction of humans who will ever exist and who will also witness the rise of AGI. We will remember what it was like when computers were dumb, artificial intelligence was pure science fiction, and humans were the smartest life forms on Earth. What we will witness will be just as consequential as the Industrial Revolution or the World Wars. Feel extremely lucky.

[wjfox]

the tiny fraction of humans

And that's such a fantastic coincidence, it's why I'm convinced we're in some kind of simulation. Maybe our experience of this pivotal time in history is ASI's way of easing us into the post-Singularity world. Sort of like a "tutorial" level before the game itself begins.

[funkervogt]

the tiny fraction of humans

And that's such a fantastic coincidence, it's why I'm convinced we're in some kind of simulation. Maybe our experience of this pivotal time in history is ASI's way of easing us into the post-Singularity world. Sort of like a "tutorial" level before the game itself begins.

That thought also occurred to me. It also militates against the argument that aliens aren't coming to Earth because they think we are boring. The transition from organic to synthetic intelligence is an incredibly rare event that can probably go off in different trajectories. Surely, aliens would want to watch us during this period. And if they are very advanced, then sending a few ships here would consume a trivial amount of their available resources.

[Vakanai]

the tiny fraction of humans

And that's such a fantastic coincidence, it's why I'm convinced we're in some kind of simulation. Maybe our experience of this pivotal time in history is ASI's way of easing us into the post-Singularity world. Sort of like a "tutorial" level before the game itself begins.

Doubtful - some population had to be alive for this transition, just because we're that population doesn't suggest any reason for this to be a simulation. Some population had to be around for the first use of fire or the first farms too.

[firestar464]

the tiny fraction of humans

And that's such a fantastic coincidence, it's why I'm convinced we're in some kind of simulation. Maybe our experience of this pivotal time in history is ASI's way of easing us into the post-Singularity world. Sort of like a "tutorial" level before the game itself begins.

That thought also occurred to me. It also militates against the argument that aliens aren't coming to Earth because they think we are boring. The transition from organic to synthetic intelligence is an incredibly rare event that can probably go off in different trajectories. Surely, aliens would want to watch us during this period. And if they are very advanced, then sending a few ships here would consume a trivial amount of their available resources.

I suppose it isn't so much that they wouldn't be interested in us, but rather they'd be so advanced that they could observe us without our knowledge. It would be silly to suppose we could observe an alien craft, much less recover a crashed one.

[funkervogt]

And that's such a fantastic coincidence, it's why I'm convinced we're in some kind of simulation. Maybe our experience of this pivotal time in history is ASI's way of easing us into the post-Singularity world. Sort of like a "tutorial" level before the game itself begins.

That thought also occurred to me. It also militates against the argument that aliens aren't coming to Earth because they think we are boring. The transition from organic to synthetic intelligence is an incredibly rare event that can probably go off in different trajectories. Surely, aliens would want to watch us during this period. And if they are very advanced, then sending a few ships here would consume a trivial amount of their available resources.

I suppose it isn't so much that they wouldn't be interested in us, but rather they'd be so advanced that they could observe us without our knowledge. It would be silly to suppose we could observe an alien craft, much less recover a crashed one.

You're probably right, though even advanced alien technology isn't perfect. Cloaking devices might sometimes briefly fail.

[firestar464]

They'd probably be advanced enough to make the craft extremely small (that's also more cost-effective) and they might be able to observe from a much greater distance.

[funkervogt]

Human evolution will proceed in two different directions:

1) Maximization of cognition. For those of us who want to stay competitive with intelligent machines. The endpoint is transforming into a giant brain floating in a jar, never sleeping, and immersed in an abstract reality where you are constantly fed problems to solve.

2) Maximization of pleasure. For those of us who want to be hedonists and are willing to be at the mercy of intelligent machines and super smart-humans. I'm less sure of the final form, but this type of human will have enhanced abilities to feel pleasure and emotions. It's wrong to assume that we, the humans of 2025, represent the limits of what can be felt and sensed.

Through a long process of successive biological modifications, it might be possible for a human alive today to become a hyper-evolved biological life form optimized for one of those functions. Though you'd be radically different at the end, you'd still have a continuity of consciousness going back to the pre-Singularity era.

By the same token, your continuity of consciousness stretches back to a time in your life when you were ignorant of multiplication and addition and when you got joy from watching cartoons. The bifurcation that I am proposing isn't that conceptually different from what naturally happens during a human's life anyway--I'm just accelerating the process with technology and taking it to its logical endpoint.

Homo sapiens like us will, in the fullness of time, be shown to occupy a middle ground of intelligence and pleasure capacity that doesn't approach the limits of either. It won't be logical for any life form to want to remain at the arbitrary point on the matrix that we inhabit any more than you would want to trade places with a person with an IQ of 75 and muted senses.

[firestar464]

Why can't we maximize cognition and pleasure at the same time?

[erowind]

Why can't we maximize cognition and pleasure at the same time?

Had this thought too, we might tie them together so that working on abstract problems is readily enjoyable.

[funkervogt]

Why can't we maximize cognition and pleasure at the same time?

Because doing one takes away from being able to do the other.

[firestar464]

Worth noting that superintelligent machines may still desire pleasure in non-problem-solving entertainment, though perhaps in ways different from how we do so.

[Vakanai]

Human evolution will proceed in two different directions:

1) Maximization of cognition. For those of us who want to stay competitive with intelligent machines. The endpoint is transforming into a giant brain floating in a jar, never sleeping, and immersed in an abstract reality where you are constantly fed problems to solve.

2) Maximization of pleasure. For those of us who want to be hedonists and are willing to be at the mercy of intelligent machines and super smart-humans. I'm less sure of the final form, but this type of human will have enhanced abilities to feel pleasure and emotions. It's wrong to assume that we, the humans of 2025, represent the limits of what can be felt and sensed.

Through a long process of successive biological modifications, it might be possible for a human alive today to become a hyper-evolved biological life form optimized for one of those functions. Though you'd be radically different at the end, you'd still have a continuity of consciousness going back to the pre-Singularity era.

By the same token, your continuity of consciousness stretches back to a time in your life when you were ignorant of multiplication and addition and when you got joy from watching cartoons. The bifurcation that I am proposing isn't that conceptually different from what naturally happens during a human's life anyway--I'm just accelerating the process with technology and taking it to its logical endpoint.

Homo sapiens like us will, in the fullness of time, be shown to occupy a middle ground of intelligence and pleasure capacity that doesn't approach the limits of either. It won't be logical for any life form to want to remain at the arbitrary point on the matrix that we inhabit any more than you would want to trade places with a person with an IQ of 75 and muted senses.

I strongly disagree. Most humans, or transhumans, posthumans, neohumans, or whatever they're called, aren't looking to fit some end point like this. The first example honestly sounds like a kind of hell, why would anyone want to be a brain in a jar forever solving problems? And to feel pleasure 24/7 forever seems pointless and just, I don't know, sad.

I think human evolution doesn't have an endpoint, much less in just two directions. I imagine there will be many, many paths, many forms that we will take. Some will live in a Sword Art Online or the Matrix style virtual reality, some will have physical forms such as cyborgs, fully robotic bodies, and genetically enhanced biological forms. Some will choose to shoot out as satellites to see the deepest reaches of the cosmos, some will just game for all eternity, many will live similar lives to now - spending time with friends, family, loved ones, just smarter and stronger and more able and capable than we can imagine. But whatever we decide to do, however we evolve, the notion that we need to fulfill some kind of function doesn't sound evolved, it sounds like the kind of primitive views we need to ascend above.

[funkervogt]

Just as human intelligence is one point on an intelligence spectrum, the first AGIs we build will also represent one point (ahead of us). As smart as they will be, it would be wrong to assume they will represent the pinnacle of what is possible. Even smarter AGIs at more distant points on the spectrum could be built with better technology and more resources.

I predict this will create a conundrum for humans in the future. The AGIs of, say, 2050 will be Godlike from our perspective, and will know everything that can be known. But among them, there will be differences in intelligence that are so high-level that humans won't be able to grasp the nature of them. AGI 1 might be smarter than AGI 2 because the former understands "Quantum Electrohydrodynamic Equation 291" while the latter does not. However, the Equation will be so complex and esoteric that no human mind could comprehend it or its significance.

Likewise, from the perspective of a house cat, Einstein was no smarter than any other human. He knew how to change the cat's water and food bowls, to empty the litter box, and to stroke the cat well. Einstein's intelligence advantage was in an abstract domain that the cat couldn't comprehend due to the limitations of its brain.

Returning to the original example, in 2050 we could be completely satisfied with our Godlike AGIs, who can answer every question and solve every problem relevant to us. However, they will tell us we need to commit more money, electricity, and land into building more data centers to support even better AGIs. When we ask what the better AGIs will be able to do that the current ones can't, the current AGIs might not be able to explain the advantage.

An authoritarian country like China would have an advantage in this scenario since its leaders would be likelier to singlemindedly pursue anything that gave them a potential advantage, even if it meant depriving the population of resources in exchange for something of unclear value. However, in democratic countries, public opinion will be much more of an obstacle. The argument that things are "good enough" could win out.

[Vakanai]

Just as human intelligence is one point on an intelligence spectrum, the first AGIs we build will also represent one point (ahead of us). As smart as they will be, it would be wrong to assume they will represent the pinnacle of what is possible. Even smarter AGIs at more distant points on the spectrum could be built with better technology and more resources.

I predict this will create a conundrum for humans in the future. The AGIs of, say, 2050 will be Godlike from our perspective, and will know everything that can be known. But among them, there will be differences in intelligence that are so high-level that humans won't be able to grasp the nature of them. AGI 1 might be smarter than AGI 2 because the former understands "Quantum Electrohydrodynamic Equation 291" while the latter does not. However, the Equation will be so complex and esoteric that no human mind could comprehend it or its significance.

Likewise, from the perspective of a house cat, Einstein was no smarter than any other human. He knew how to change the cat's water and food bowls, to empty the litter box, and to stroke the cat well. Einstein's intelligence advantage was in an abstract domain that the cat couldn't comprehend due to the limitations of its brain.

Returning to the original example, in 2050 we could be completely satisfied with our Godlike AGIs, who can answer every question and solve every problem relevant to us. However, they will tell us we need to commit more money, electricity, and land into building more data centers to support even better AGIs. When we ask what the better AGIs will be able to do that the current ones can't, the current AGIs might not be able to explain the advantage.

An authoritarian country like China would have an advantage in this scenario since its leaders would be likelier to singlemindedly pursue anything that gave them a potential advantage, even if it meant depriving the population of resources in exchange for something of unclear value. However, in democratic countries, public opinion will be much more of an obstacle. The argument that things are "good enough" could win out.

Of course this is making a few assumptions. One, that an AGI will tell us we need to deprive ourselves and other humans of resources to make an even more intelligent AGI, and second that there is an actual advantage for us in doing so. If an AGI prioritizes intelligence for intelligence's sake alone, then we might wind up depriving people of better lives solely to satiate an AGI's need to know, in which case we do not receive any advantage from this. In that scenario China might actually wind up worst off as people would suffer just for a smarter AGI that doesn't benefit them.

We can't villify the "good enough" argument, because like anything it's all a cost vs benefit argument. What good is great cost for little to no benefit? If we fail to properly align AGI we might wind up in a scenario where it will view us as completely expendable to it's pursuit of creating an even smarter AGI. Should we lead ourselves to extinction, or near to it, to satisfy the urge for a greater AGI?

Ultimately I feel the pursuit of intelligence at this scale needs to be done for our benefit, lest we risk pursuing it towards our detriment.