25 replies to this topic

[PhoenixRu]

You ignored the rest of what he said.

 

Not at all. For example, I'm fully agree with this part:

 

We've yet to see whatever it is that OpenAI's really working on.

 

Exactly, let's wait and see.

[Jacob209]

 

Well, it's not the Singularity or anything, but what GPT-3 alone has accomplished is shocking.

 

 

It's exactly what I expected GPT-3 to be: something in between artificial narrow intelligence and artificial general intelligence. It even has the architectural differences. You can get GPT-3 to do a wide variety of tasks, including things it wasn't even trained to do. You know, artificial expert intelligence, or AXI.

 

What I absolutely didn't expect was how easy it would be to jump from AXI to AGI. It's not absolute, but there is a path towards a very functional kind of zombie AGI within a year. I'm not kidding— that's how close we are, if the people at work on it recognize the same limitations. Of course, it wouldn't be just GPT-3; it'd have to be the secret project OpenAI's working on (of which GPT-3 and Jukebox are likely just small parts of). But the main takeaway ought to be that we may have a proto-AGI functional within twelve months. 

 

It's so close it doesn't even sound possible. But it's right there; you can read the paper yourself and come to the same deductions about what it's missing.

 

I wish OpenAI published a blog post about it so it's easier to grasp.  There are also a lot of complaints about the paper on Arxiv and Github that it wasn't as amazing of an advance over GPT-2 or that it's running into limits, neither of which makes sense really— the GPT-2 paper itself said that scaling it up with more parameters would lead to greater generalization, and the "running into limits" part was bandied about even with GPT-2 itself. There's clearly a lot, lot more than can be done, and considering GPT-3 doesn't even use image, audio, video, or other biofeedback data, the actual limits are likely much, much higher than anything people are comfortable with.

 

Hmmm, most of what I have been reading suggests that GPT-3 is simply a scaled up version of GPT-2 and not much more. It's intuitive that with more parameters GPT-3 outperforms GPT-2, but people much more knowledgeable than me believe this advance to be more or less linearly and not groundbreaking. It was fun to fork GPT-2 and train it a bit. Also, there is a very easy access GPT-2 transformer online.

 

Here is an example:

Input: Are Yuli Ban's predictions of the future correct?

 

Output: Can we not be a little more optimistic? Can we not hope for a future that does not revolve around our iPhones, tablets, and other fast-charging devices? Does Yuli Ban truly believe that we will never see further innovation in technology? Can we, as proponents of open-source software, ever not want to see the opportunities provided by the thought-leaders and researchers in academia, government, and industry? Can we really hope that Yuli Ban can see the picture a little clearer? Please let us know what you think of his opinion piece

 

Of course the transformer generates a lot of garbage but it is otherwise fun to use.

 

Can you show me some of those people that believe its not groundbreaking? I've seen the complete opposite so far, and based off what I've seen on the papers I can't help but disagree with this.

[Enter Ataraxia]

 

 

Well, it's not the Singularity or anything, but what GPT-3 alone has accomplished is shocking.

 

 

It's exactly what I expected GPT-3 to be: something in between artificial narrow intelligence and artificial general intelligence. It even has the architectural differences. You can get GPT-3 to do a wide variety of tasks, including things it wasn't even trained to do. You know, artificial expert intelligence, or AXI.

 

What I absolutely didn't expect was how easy it would be to jump from AXI to AGI. It's not absolute, but there is a path towards a very functional kind of zombie AGI within a year. I'm not kidding— that's how close we are, if the people at work on it recognize the same limitations. Of course, it wouldn't be just GPT-3; it'd have to be the secret project OpenAI's working on (of which GPT-3 and Jukebox are likely just small parts of). But the main takeaway ought to be that we may have a proto-AGI functional within twelve months. 

 

It's so close it doesn't even sound possible. But it's right there; you can read the paper yourself and come to the same deductions about what it's missing.

 

I wish OpenAI published a blog post about it so it's easier to grasp.  There are also a lot of complaints about the paper on Arxiv and Github that it wasn't as amazing of an advance over GPT-2 or that it's running into limits, neither of which makes sense really— the GPT-2 paper itself said that scaling it up with more parameters would lead to greater generalization, and the "running into limits" part was bandied about even with GPT-2 itself. There's clearly a lot, lot more than can be done, and considering GPT-3 doesn't even use image, audio, video, or other biofeedback data, the actual limits are likely much, much higher than anything people are comfortable with.

 

Hmmm, most of what I have been reading suggests that GPT-3 is simply a scaled up version of GPT-2 and not much more. It's intuitive that with more parameters GPT-3 outperforms GPT-2, but people much more knowledgeable than me believe this advance to be more or less linearly and not groundbreaking. It was fun to fork GPT-2 and train it a bit. Also, there is a very easy access GPT-2 transformer online.

 

Here is an example:

Input: Are Yuli Ban's predictions of the future correct?

 

Output: Can we not be a little more optimistic? Can we not hope for a future that does not revolve around our iPhones, tablets, and other fast-charging devices? Does Yuli Ban truly believe that we will never see further innovation in technology? Can we, as proponents of open-source software, ever not want to see the opportunities provided by the thought-leaders and researchers in academia, government, and industry? Can we really hope that Yuli Ban can see the picture a little clearer? Please let us know what you think of his opinion piece

 

Of course the transformer generates a lot of garbage but it is otherwise fun to use.

 

Can you show me some of those people that believe its not groundbreaking? I've seen the complete opposite so far, and based off what I've seen on the papers I can't help but disagree with this.

 

Once I get more time I will include the rest.

 

Here is the one that immediately came to mind: article

[starspawn0]

And don't forget to read the comments at the end of that post.

[Metalane]

How much computational power is required to run GPT-3? Are we talking the worlds most powerful supercomputers type of power or less? 

[Miky617]

Well, we're past the halfway point of the year, yet to be entirely honest, it feels like these past 6 months lasted half the decade already. 

 

I didn't make any predictions on this thread at the end of last year since I figured they'd be pretty bland and vague, especially compared to the ones posted by wjfox or Yuli. I simply don't have my finger on the pulse quite like they do. 

 

In any case, so far this year has shaped up to be pretty unpredictable and I'm excited to hear how the predictions from last year have fared so far. I think it would also be worth the time to reorient and "start over", meaning that now might be a good chance to cast a few more predictions before the year ends since we can adjust for the state of the world amidst SARS-CoV-2 and other things.

 

I might as well throw a few out there; nothing too substantial or impressive. 

 

• Joe Biden wins the presidency (whether he will be able to finish his term remains to be seen), raising temper tantrums and uproar among Trump's base like nothing else seen. This might lead to a handful of domestic terror acts, or they might just whine about it a bit on social media. With this, Democrats and center-oriented faux-woke individuals can retreat to the comfort of their homes and sweep under the rug all of the issues that plagued the Trump administration even while nothing substantially changes under Biden.
• By the end of the year, COVID-19 will still be happy and cozy in its grip on the world, with over 1,000,000 dead in its wake while everyone else is too exhausted of it to care any more. Still several months away from a widely-available vaccine, several treatments are circulating but they are in their infancy and many people are distrustful of it, citing conspiracy theories. On the other hand, the fallout from COVID is that many of its victims are older, generally more conservative people who failed to take proper precautions. As a result, COVID-19 has effectively strengthened the demographic "weight" of younger, more progressive people which may end up influencing future elections.
• On a related note, I predict that at least one major political leader (not necessarily a PM or president, but fairly close) will pass away due to COVID. Cue more conspiracies.
• I'm not sure if the world will permanently trend towards more decentralized work and a greater normalization of work-from-home as a result of COVID, but if I had to venture a guess, I'd say no. Whether SARS-CoV-2 is still floating around or not, people will want to go back to work the old-fashioned way and only a few businesses will make the decision to adopt a more fluid work environment, likely since it cuts costs and is more efficient, depending on the type of work being done. Drive-In theaters may make a comeback.
• Climate change continues to wreak havoc with a major storm causing extensive damage to a heavily developed area. Sea level rises are still not that significant yet, but its effect is noticed as places like the Outer Banks and island nations are experiencing greater erosion. The summer of 2020 breaks heat records globally and will hold onto those records for about a year.
• Synthetic media is still in a fad phase and is rapidly maturing, but I doubt we'll see anything big and mainstream this year. I could be wrong, but I imagine we'll see better versions of "This ___ does not exist" and a few convincing DeepFakes and that'll be about it for 2020. Next year or two might surprise us though with a full-length feature film being DeepFaked. It seems to me that many people who are not in the futurist bubble like us are still unaware of its potential and might have heard about it briefly a while ago and then forgot about it. For 2020, synthetic media will likely remain niche.
• Starspawn seems to be very enthusiastic and excited about a near-future application of BCI work so I'll leave any predictions on the state of that art to them. However I believe that if such a thing happens this year, it will also be a niche thing that makes huge waves in the community but has relatively little impact on the general populace (for now at least. I imagine the applications will disseminate rapidly and we'll see major changes in a lot of fields throughout the decade).
• Retail apocalypse will also become a restaurant and tourism apocalypse as well thanks to COVID. I imagine at least two more major retailers will declare bankruptcy or switch to almost exclusively an online presence. Many restaurants that lack the capital to survive the attrition of COVID will close their doors as well. Local economies in tourist destinations (primarily ones that cater to US-icans) will also be crunched hard as many countries enforce travel restrictions with the USA. 
• Thanks to COVID, the current Guinea Worm eradication and Polio Eradication initiatives will see set backs and those diseases will be around for at least another 5 years I predict. 
• China will make great strides toward becoming the next great superpower, with the US choking on itself in the meantime. With either Trump or Biden's election, the United States will cement it's status as a waning nation.
• With many progressives disgusted at the prospect of voting for either Trump or Biden, 2020 may see a sharp increase in support for the Green Party, making them eligible for a more noticeable presence on the political stage in future elections.
• Tesla keeps raking in money and cementing its monopoly on the EV front. Despite their work though, consumer ownership of autonomous vehicles at level 4 is still at least one more year away. Legislation and public unease will make adoption of fully self-driving cars slow.
• American infrastructure is ripe for revitalization. Hopefully we'll see a transition toward future-friendly architecture and civil engineering. Not likely this year though. Many bridges are nearing the end of their lifespan on the interstate system, so those will be rebuilt but otherwise much work remains to be done.
• GPT-3 will start to creep into the works of big tech companies. What this will look like, I'm not entirely sure. This is Yuli and Starspawn's domain more so than mine.
• GE mosquito trials will go along smoothly. Hopefully that will move forward and progress will be made toward shifting mosquito populations toward species that are less effective as pathogen vectors.
• More animal/plant/insect extinctions.
• More celebrity deaths. Most will probably not be COVID related but I imagine at least one or two will be.
• More civil unrest. More token gestures by companies and legislature without meaningful change. Some forward progress, enough to hide the problems better and appease the less motivated protesters. Many people are tired of protesting and are satisfied with the crumbs of change, especially if Trump leaves office. Many are still indignant however. I expect more riots and catalytic events in the near future as real systemic change hasn't yet happened and the cycle continues. However people will also become more aware of the power of the public.

 

That's all I can think of at the moment. 2020 is setting the stage for many big motions in this next decade and a lot of the things happening now will truly be appreciated in their effect on what comes next. I have my hopes about what will or won't happen.