Jump to content

Welcome to FutureTimeline.forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. If you already have an account, login here - otherwise create an account for free today!
Photo

Your 2020 Predictions

2020 technology society politics

  • Please log in to reply
25 replies to this topic

#1
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,318 posts
  • LocationNew Orleans, LA

Your 2015 Predictions
Your 2016 Predictions
Your 2017 Predictions
Your 2018 Predictions
Your 2019 Predictions
Your 2020 Predictions ←You are here!
 
It's that time of the year!

At long last, we've arrived at AD 2020. Now it's time to predict what you think will happen in the new decade's inaugural year.


And remember my friend, future events such as these will affect you in the future.


#2
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,318 posts
  • LocationNew Orleans, LA

Main Timeline predictions

  • Microsoft ends support for Windows 7
  • America's power shift is destabilising the Asia-Pacific region
  • Generation X is reshaping global politics
  • Internet use reaches 5 billion worldwide
  • Texting by thinking
  • Complex organ replacements grown from stem cells
  • The first stem cell therapy for congestive heart failure
  • A cure for malaria
  • Progress with longevity extension
  • Genetically engineered "super" bananas
  • Tokyo hosts the Olympic Games
  • Hong Kong's Mass Transit Railway (MTR) has been significantly expanded
  • Completion of the Fehmarn Belt Fixed Link | The UK has expanded its offshore grid connections
  • Smart meters in every UK home | Mercury pollution has been greatly reduced
  • Glacier National Park and other regions are becoming ice-free
  • Britain's new aircraft carriers reach full operational capability
  • 30,000 drones are patrolling the skies of America
  • Mars 2020 rover mission
  • The first test flights of NASA's Quiet Supersonic Technology
  • England's Coastal Path is open to walkers
  • Expo 2020 is held in Dubai
  • The PlayStation 5 is launched
  • The final collapse of the Larsen B ice shelf

My Predictions
 
Disclaimer: Due to my atrocious batting average with them, I'm completely excluding any geopolitical predictions, save for those related to technology in some way.
 

  • Virtual reality receives its first "killer app" that specifically drives sales of headsets
  • The second generation of VR headsets are formally teased, with the first likely receiving a Q4 2021 or unknown-quarter 2022 release date.
  • Apple unveils an augmented/mixed reality headset, though for a very high price and marketed more towards enterprise & business. The form factor is considered a massive step forward from earlier designs (such as HoloLens and Magic Leap)
  • 5G coverage expands, particularly across China, the wealthier East Asian countries, and Europe.
  • With 5G comes an explosion of Internet of Things gadgetry, plenty of which will remain overpriced and unnecessary due to the prevalence of smartphones and smart speakers
  • Smart speakers will begin to be fused with robotic apparatuses that allow them to move about or even physically manipulate certain objects, ushering a second wind for domestic robotics after the disastrous first wave of 2016-2019. Think of video game console add-ons, but now for Amazon Echo and Google Home devices.
  • Starlink and other global internet services will begin engaging in active operations. Satellites will be the dominant method, followed by high-altitude balloons and drones.
  • Passenger drones will begin carrying their first passengers as part of "flying taxi" services, likely in Dubai and certain East Asian cities
  • Psilocybin is decriminalized or legalized in a US state
  • Marijuana legalization continues to spread. Mexico likely legalizes, followed by a European and Asian country. At least 30 US states have either legal or decriminalized weed. Also, a southern US state will send to a referendum, with the answer being to legalize by 2022. Also, a southern US state will send to a referendum, with the answer being to legalize by 2022.
  • The first public commercial sale of cultured meat occurs this year.
  • Metafarming (including vertical farming, urban farming, and hydroponics) will see massive gains, especially in response to a growing need for solutions to the effects of climate change
  • Geoengineering proposals will slowly enter mainstream discourse
  • CRISPR is used once more for the purpose of genetically modifying a human embryo, though not for the purpose of creating a designer baby a la Lulu & Nana.
  • Mammoth genetic material is able to be inseminated into an elephant, but no birth is undertaken
  • Level 3 autonomy becomes a more widespread feature of cars
  • Public tests of Level 4 AVs increase in number, but there are no commercially available Level 4 AVs
  • Number of electric cars on the road passes 10 million worldwide
  • Tesla fails to achieve profitability in more than one quarter, but its stock price reaches at least $500 a share
  • SpaceX's first manned human space flights occur
  • Solar power passes 800 GW of installation worldwide
  • Sustained nuclear fusion with net energy is achieved, but plans for fusion power remain in the future
  • A room-temperature superconductor is formally found, perhaps with temperatures up to 50­°F. However, it is only possible at extreme pressures, perhaps 200 gigapascals.
  • Quantum supremacy is formally achieved (sans skepticism)
  • DeepMind's newest network demonstrates the ability to play NES games as well as Atari games, chess, go, and shogi.
  • The first discussions of a need to create a new definition for AI architecture to handle the burgeoning "in between ANI and AGI" capabilities of generalized narrow intelligence will be passed around
  • Boston Dynamics reveals a new robot that is non-humanoid (perhaps spider/octopode-esque) in design
  • The first commercially available bipedal humanoid robot will be released for business use, most likely in China and Japan (most likely the UBTech Walker)
  • Donald Trump will announce a robotics and AI initiative to counter China's growing AI dominance. This will likely result in a new government agency, "Department of Machine Intelligence" or something along those lines.
  • Privacy has emerged as a primary political issue. The virtually constant use of electronic communication technologies is leaving a highly detailed trail of every person’s every move.
  • Artificial neural networks as used for media synthesis continue to improve exponentially: "This [X] Does Not Exist" will boom in number
  • Deepfakes will be successfully used for a billion-dollar scam
  • Human image synthesis becomes indistinguishable from a photograph
  • A natural language generation (NLG) transformer will pass the Turing Test by a shocking amount (but obviously will not herald artificial general intelligence)
  • OpenAI's GPT-2 will be expanded to at least 15 billion data parameters
  • A Grover AI-generated article will be retweeted at least 100,000 times, including despite being exposed as AI-generated
  • Neural style transfer will be applied to music (i.e. shifting a guitar into a synth; a drum into a piano) as part of either a popular or memetic app/website (by "memetic" I mean that it mainly attains popularity and wide discussion for a relatively short period of time, such as a few weeks, before largely falling under the radar; sort of like DeepNudes or ThisPersonDoesNotExist).
  • Deepfake videos & images will be retweeted and reposted on social media upwards of 2+ million times in the lead-up to the 2020 US Presidential Election
  • A new method for creating deepfakes will reduce the ease in their creation to the point that it will be possible to release an app dedicated to things such as full-video face-swapping.
  • Artificial news anchors are used in increasing numbers in China
  • NLG chatbots will begin being used online and outperform traditional chatbots by cosmic amounts
  • Internet comment sections will begin being infested by bots utilizing a combination of NLG for text and image synthesis for profile images.
  • An NLG-based "stylistic editor" will be tested online, allowing a passage of text to be rewritten/edited in the style of another writer (or just in general)
  • A new subreddit will be created that allows for human users & traditional bots to interact with NLG bots, as an evolution of /r/SubSimulatorGPT2.
  • Image manipulation/enhancement is used to upscale video game graphics, essentially allowing for a 6th gen game to have seemingly 8th gen graphical fidelity (if the textures are manually modded into the game)
  • Simulated bots allow for "virtual actors" in things like Minecraft or Steam Filmmaker. These will follow basic commands thanks to natural language processing.
  • Image synthesis will begin giving way to video synthesis; at least 10-second long clips will be feasible, allowing to create a mostly photorealistic gif of things that do not exist, which may be used to create This Gif Does Not Exist
  • Conversely, the ability to animate still images (that show some evidence of action) will become available online, which might also be the basis for This Gif Does Not Exist
  • The first generation of non-invasive portable MRIs are released, allowing researchers to reduce the brain to easily studiable voxels
  • An EEG BCI is used to type 100+ alphanumeric characters per minute, effectively allowing texting-by-thinking
  • Brain-scan & eye-tracking data fed into neural networks allows for explosively massive gains, perhaps upwards of two orders of magnitude over traditional methods providing there are enough scans. This may be used for additional power to NLG, driverless car improvements, and autosummarization, among others, but don't expect too much out of this just yet
  • Conversely, machine learning-enhanced EEGs may be shown to allow for practical neurocontrollers in video games/virtual reality/augmented reality
  • An update will be released on invasive BCIs, showing considerable progress since the announcement of Neuralink, but nothing yet able to translate into practical use; the first possible use-cases for medical research are predicted to be at least five to ten years away

And remember my friend, future events such as these will affect you in the future.


#3
10 year march

10 year march

    Member

  • Members
  • PipPipPipPipPip
  • 343 posts
US chinese tensions intensify as China's nominal gdp continues to close in on the US's nominal gdp.

A boring democrat is nominated and Trump wins the election.

Roughly 4 small but noticeable breakthroughs in Ai occur the significance of these breakthroughs is roughly similar to the significance of the alpha go breakthrough.

In my personal life 2020 marks a year of growth In many areas coming out of a period of personal recession which took place in the prior 2 years.

Speaking of recession I expect the US 2020 recession will begin this year.

#4
funkervogt

funkervogt

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,083 posts

An AI is among the top ten best StarCraft 2 players. 

 

A disappointing person wins the U.S. Presidential election. 



#5
TranscendingGod

TranscendingGod

    2020 is here; I still suck

  • Members
  • PipPipPipPipPipPipPipPip
  • 2,021 posts

Well, my 2019 prediction was off the mark because I underestimated the depth of problems plaguing the United States.

 

I don't have a particular prediction for 2020 only many vague ideas about the general direction of things. I would like to remark that if Yuli's prediction about fusion energy comes to pass 2020 will go down as one of the most momentous years in human history. Like the years when Einstein published his seminal papers on relativity. 


The growth of computation is doubly exponential growth.

#6
TranscendingGod

TranscendingGod

    2020 is here; I still suck

  • Members
  • PipPipPipPipPipPipPipPip
  • 2,021 posts

Well, my 2019 prediction was off the mark because I underestimated the depth of problems plaguing the United States.

 

I don't have a particular prediction for 2020 only many vague ideas about the general direction of things. I would like to remark that if Yuli's prediction about fusion energy comes to pass 2020 will go down as one of the most momentous years in human history. Like the years when Einstein published his seminal papers on relativity. 

I spoke too soon. https://www.wsj.com/...ars-11580398277

I was right. 


The growth of computation is doubly exponential growth.

#7
Miky617

Miky617

    Member

  • Members
  • PipPipPip
  • 54 posts
  • LocationNC

Well we've made it to June. Is it time for our halfway progress report?



#8
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,318 posts
  • LocationNew Orleans, LA

I'll do a midyear update on the first of July (when six months have actually passed).

 

As a preview: I'm in utter shock with how I underestimated the progress in artificial intelligence. Something weird is going on, even by the standards of the past decade.

 

I'm also deeply glad I didn't do any geopolitical or social predictions. I would not have gotten a damn thing.


And remember my friend, future events such as these will affect you in the future.


#9
PhoenixRu

PhoenixRu

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,062 posts

As a preview: I'm in utter shock with how I underestimated the progress in artificial intelligence. Something weird is going on, even by the standards of the past decade.

 

With all this coronavirus / geopolitical stuff, I completely missed this topic. The only "weird thing is going on" I noticed was the new law regulating the AI-related matters passed by State Duma in april. As far as I remember, my thoughts were "WTF? Is this the right time to discuss AI?"

 

I'm also deeply glad I didn't do any geopolitical or social predictions. I would not have gotten a damn thing.

 

Yeah, the first half of the year was crazy. I have some ideas about the second half, but will refrain from posting them here.



#10
eacao

eacao

    Lobster

  • Members
  • PipPipPipPipPip
  • 477 posts
  • LocationAustralia

I'm in utter shock with how I underestimated the progress in artificial intelligence. Something weird is going on, even by the standards of the past decade.

 

Seconded. Why haven't you explained all this? You're usually the one explaining AI.


If you're going through hell, keep going. - Winston Churchill

You don't decide your future. You decide your habits, and your habits decide your future.
Nearly all men can stand adversity, but if you want to test a man's character, give him power. - Abraham Lincoln.


#11
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,318 posts
  • LocationNew Orleans, LA

 

I'm in utter shock with how I underestimated the progress in artificial intelligence. Something weird is going on, even by the standards of the past decade.

 

Seconded. Why haven't you explained all this? You're usually the one explaining AI.

Well, it's not the Singularity or anything, but what GPT-3 alone has accomplished is shocking.

It's exactly what I expected GPT-3 to be: something in between artificial narrow intelligence and artificial general intelligence. It even has the architectural differences. You can get GPT-3 to do a wide variety of tasks, including things it wasn't even trained to do. You know, artificial expert intelligence, or AXI.

 

What I absolutely didn't expect was how easy it would be to jump from AXI to AGI. It's not absolute, but there is a path towards a very functional kind of zombie AGI within a year. I'm not kidding— that's how close we are, if the people at work on it recognize the same limitations. Of course, it wouldn't be just GPT-3; it'd have to be the secret project OpenAI's working on (of which GPT-3 and Jukebox are likely just small parts of). But the main takeaway ought to be that we may have a proto-AGI functional within twelve months. 

 

It's so close it doesn't even sound possible. But it's right there; you can read the paper yourself and come to the same deductions about what it's missing.

 

I wish OpenAI published a blog post about it so it's easier to grasp.  There are also a lot of complaints about the paper on Arxiv and Github that it wasn't as amazing of an advance over GPT-2 or that it's running into limits, neither of which makes sense really— the GPT-2 paper itself said that scaling it up with more parameters would lead to greater generalization, and the "running into limits" part was bandied about even with GPT-2 itself. There's clearly a lot, lot more than can be done, and considering GPT-3 doesn't even use image, audio, video, or other biofeedback data, the actual limits are likely much, much higher than anything people are comfortable with.

 

 

As a preview: I'm in utter shock with how I underestimated the progress in artificial intelligence. Something weird is going on, even by the standards of the past decade.

 

With all this coronavirus / geopolitical stuff, I completely missed this topic. The only "weird thing is going on" I noticed was the new law regulating the AI-related matters passed by State Duma in april. As far as I remember, my thoughts were "WTF? Is this the right time to discuss AI?"

 

I'm also deeply glad I didn't do any geopolitical or social predictions. I would not have gotten a damn thing.

 

Yeah, the first half of the year was crazy. I have some ideas about the second half, but will refrain from posting them here.

I didn't even catch that. I've been outright meaning to ask you if the Russian government or if the other people you hang out with on Russian language forums are in any way taking AI more seriously. This is absolutely the time (or at the very least, we're very close to it), and pray tell, I'm not the right person to explain why.

 

As for geopolitics...

 

The first half of the year was beyond prediction. A lot of the chaos was rooted in a black swan event. The riots and protests and "proto-maidan" in the USA were exacerbated by the shutdown and economic fallout. But that doesn't make it any less legitimate. Perhaps the masters kept the gunpowder wet for decades, and now that it's dried up, it was inevitable. And it's not like things can't get even sillier. 

 

twoday (from another forum)
a brief history lesson:
 
After the Netherlands became independent, the Dutch Republic soon emerged from being a feudal possession of the Spanish to becoming a flourishing wealthy empire. Free from the system of monarchy which burdened the rest of the countries in Europe, the country soon became an oligarchy run by wealthy merchants who also ran the government, and went to work building a mighty fleet that they used to enable their imperialist expansion to every continent and all corners of the world, ruthlessly exploiting every resource they came across for the sake of accumulating wealth. Not only did they invent the stock market and the concept of the corporation, but they also seamlessly integrated corporate and government interests into a single ruthless machine, and also established a global trade network which arguably played a massive part in establishing the basis for the current world order of international capitalist trade. In doing so, they rapidly accumulated unfathomable amounts of wealth, and the profits of the East India Company alone dwarf all modern ones when adjusted for inflation, and made it the most profitable company in the history of the world:
 
WxWUTYE.png
 
And that's just ignoring the West India Company and all of the other business which played a role in the economy. For a number of decades in the 17th century the Dutch Republic was by any measure the wealthiest nation on earth, and much of this money was invested into the armed forces and military technology, resulting in one of the most formidable armies and navies on earth, which was then used to build an exploitative global empire that drained the earth of its natural resources in the name of capital, and led to deaths of hundreds of thousands of people who stood in the way of this process. There was a period where the Dutch were pretty much the only global superpower, and not only dominated the world financially and militarily, but also created lots of scientific developments and beautiful artwork. So what happened? Why don't the Dutch still rule the world?
 
Well, all of this wheeling and dealing inevitably led to the Dutch becoming embroiled in numerous conflicts around the world with all of the other major powers of the day, and those decades of war (both cold and hot) drained a lot of resources and created a lot of enemies on the world stage. These wars were not only a financial burden on the nation, but also helped emphasize social divides which were already present, and generated a lot of problems in society. All of this came to a head in a single year which is now known as the 'Rampjaar' or "Disaster Year," when just about everything that could possibly go wrong went wrong. After years of conflict, the other major powers all grew to despise the Dutch Republic, and in 1672 the country was simultaneously attacked on all sides by an Alliance of the English, French, and Germans. At the same time there was a bitter civil war in the country between monarchists and Republicans which culminated in the head of state, who had ruled for almost 2 decades, and his brother who was the head of the navy (perhaps the most formidable military force on earth at that time) being dragged out into the streets, where they were lynched by a mob who ripped off their skin and ate their livers and other organs raw.
 
TNRUnmq.jpg
 
This mighty empire descended almost overnight into unimaginable chaos, with every element of society being ripped apart along its weakest seams. There is a phrase from this time which describes the state of the nation - "radeloos, redeloos, reddeloos," which translates into something like "without leadership, without reason, and without possibility of salvation." Though they were able to fight off the armies which threatened to wipe them off the map, the economy was hopelessly devastated and would remain so for centuries. A leadership crisis emerged, and for the next hundred years the country was ruled by the nephews and cousins of previous rulers, who had grown up fat and wealthy and had  no idea how to govern what remained of the country. While the average citizen became horribly poor and hopeless, the ineffective leaders continued to live in mansions and eat off of silver plates, and they turned to increasingly nefarious tactics to prop up their economy. They magnified the slave trade a thousand fold, and started using increasingly brutal tactics to retain control over their colonies, which were taken away from them by their enemies one by one. They invested in pet projects and other tremendous wastes of money which did little to help the country or its economy. For instance, the remaining few Dutch of this period who still had some wealth spent a lot of it financing the rebels in America in the late 1700's, largely out of spite against their ancient enemies, the British. But it didn't help, of course. They were invaded by the French and Germans (and English and even Russian armies at some point), and the nation continued along its disastrous trajectory, leading to revolution, invasion, poverty, and death. Lots of death. Many such horrors. The reason why so many Dutch cities have charming 17th century buildings which attract tourists today is that they never recovered economically from the Disaster Year until relatively recently, and for hundreds of years there was little money around for new building projects, and everyone just had to continue living in the houses which were built way back when during "the Golden Age."
 
So what's the point of this story? There was once a global superpower which was wealthier than any other in history, with a massive and powerful army which it used to terrorize the world, as well as form much of the global capitalist economic system which surrounds us today. But then one day disaster struck and the chickens came home to roost, and it all fell apart overnight. Poof. That's all it takes. And there's a good chance that anyone who is reading this post didn't know any of this. That entire empire so thoroughly destroyed and forgotten (and good riddance), that now, 350 years later very few people in the world know that it ever happened.
 
The moral of the story is that no matter how wealthy, how powerful the empire which you inhabit is, it is ultimately nothing within the grand scheme of history. Your entire world can just collapse one day, and then a few hundred years later barely anyone alive will know anything about it. Everything which surrounds you and all that you know is ultimately just an insignificant blip on the radar of human progress that can and will be forgotten. All it takes is one strong shock which tears everything apart. And that's what we are witnessing happening in much of the Western world right now. So good luck to everyone with all of that.

And remember my friend, future events such as these will affect you in the future.


#12
PhoenixRu

PhoenixRu

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,062 posts

I didn't even catch that. I've been outright meaning to ask you if the Russian government or if the other people you hang out with on Russian language forums are in any way taking AI more seriously. This is absolutely the time (or at the very least, we're very close to it), and pray tell, I'm not the right person to explain why.

 

Very briefly, from what I see in Russian magazines/forums:
 
- there is still no visible road from the current quasi-AI to the real strong AI.
- therefore, people do not expect to see the strong AI in the near future.


#13
Enter Ataraxia

Enter Ataraxia

    Member

  • Members
  • PipPip
  • 44 posts
  • Location+0° 2′ 46.2″, 179° 56′ 39.4″ 7.9E2+/-420parsecs

Thank you for posting these predictions, I really enjoyed reading them. As someone who employs machine learning models, it is nice to see mention of areas that I have not formally looked into. I want to read more DeepMind and OpenAI blog entries and research papers.


"Utopia is the hope that the scattered fragments of good that we come across from time to time in our lives can be put together, one day, to reveal the shape of a new kind of life. The kind of life that yours should have been." - Bostrom

 


#14
Enter Ataraxia

Enter Ataraxia

    Member

  • Members
  • PipPip
  • 44 posts
  • Location+0° 2′ 46.2″, 179° 56′ 39.4″ 7.9E2+/-420parsecs

Well, it's not the Singularity or anything, but what GPT-3 alone has accomplished is shocking.

 

 

It's exactly what I expected GPT-3 to be: something in between artificial narrow intelligence and artificial general intelligence. It even has the architectural differences. You can get GPT-3 to do a wide variety of tasks, including things it wasn't even trained to do. You know, artificial expert intelligence, or AXI.

 

What I absolutely didn't expect was how easy it would be to jump from AXI to AGI. It's not absolute, but there is a path towards a very functional kind of zombie AGI within a year. I'm not kidding— that's how close we are, if the people at work on it recognize the same limitations. Of course, it wouldn't be just GPT-3; it'd have to be the secret project OpenAI's working on (of which GPT-3 and Jukebox are likely just small parts of). But the main takeaway ought to be that we may have a proto-AGI functional within twelve months. 

 

It's so close it doesn't even sound possible. But it's right there; you can read the paper yourself and come to the same deductions about what it's missing.

 

I wish OpenAI published a blog post about it so it's easier to grasp.  There are also a lot of complaints about the paper on Arxiv and Github that it wasn't as amazing of an advance over GPT-2 or that it's running into limits, neither of which makes sense really— the GPT-2 paper itself said that scaling it up with more parameters would lead to greater generalization, and the "running into limits" part was bandied about even with GPT-2 itself. There's clearly a lot, lot more than can be done, and considering GPT-3 doesn't even use image, audio, video, or other biofeedback data, the actual limits are likely much, much higher than anything people are comfortable with.

 

Hmmm, most of what I have been reading suggests that GPT-3 is simply a scaled up version of GPT-2 and not much more. It's intuitive that with more parameters GPT-3 outperforms GPT-2, but people much more knowledgeable than me believe this advance to be more or less linearly and not groundbreaking. It was fun to fork GPT-2 and train it a bit. Also, there is a very easy access GPT-2 transformer online.

 

Here is an example:

Input: Are Yuli Ban's predictions of the future correct?

 

Output: Can we not be a little more optimistic? Can we not hope for a future that does not revolve around our iPhones, tablets, and other fast-charging devices? Does Yuli Ban truly believe that we will never see further innovation in technology? Can we, as proponents of open-source software, ever not want to see the opportunities provided by the thought-leaders and researchers in academia, government, and industry? Can we really hope that Yuli Ban can see the picture a little clearer? Please let us know what you think of his opinion piece

 

Of course the transformer generates a lot of garbage but it is otherwise fun to use.


"Utopia is the hope that the scattered fragments of good that we come across from time to time in our lives can be put together, one day, to reveal the shape of a new kind of life. The kind of life that yours should have been." - Bostrom

 


#15
PhoenixRu

PhoenixRu

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,062 posts

It was fun to fork GPT-2 and train it a bit.

 

As far as I remember, there is a whole thread dedicated to such generated texts. Some of them were amazingly coherent, and some were amazingly hilarious.

 

The pearl of wisdom from GPT2. I myself couldn't say better:

 

Predicting the scientific and socio-political events of 2020 is not an easy task: a lot can happen between now and then. I will write about some major events, but note that nothing is finalized and nothing is certain at all at the moment. If you wish to share your opinion on the future of science, technology, society, environmentalism, politics, or whatever, you can do



#16
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,318 posts
  • LocationNew Orleans, LA

As I've mentioned before, the issue with saying "GPT-3 is just a bigger GPT-2" is that GPT-2 was already amazing. With the right training, it was able to even generate rudimentary images and music and even play chess. What's more, OpenAI's original GPT-2 paper outright said that adding more parameters would lead to greater generalization. There's nothing that's happening now that wasn't predicted.

 

Furthermore, it's become increasingly clear that GPT-3 (as well as Jukebox) is just the appetizer. We've yet to see whatever it is that OpenAI's really working on.


And remember my friend, future events such as these will affect you in the future.


#17
PhoenixRu

PhoenixRu

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,062 posts

GPT-2 was already amazing.

 

Amazing, but still nothing more than a text generator, which pulls the "appropriate" words and phrases out its growing database without real understanding of context. Yes, the neural network is "learning" and database is growing, but this is not real AI (at least as I understand it). There are still many years ahead before some GPT-8 will write its first readable procedurally generated book.

 

But with music, things are much better: AI "composers" are already indistinguishable from Beethoven or Mozart.



#18
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPipPip
  • 2,087 posts

I think people are underestimating how big an advance GPT-3 is, just like they underestimated the arrival of next-gen BCIs.  

 

Now, to be sure, it's not an advance on the "theory" side.  It appears not to add anything new, as far as architecture, training, data curation, or testing are concerned.  So, you'll probably see people complain, "Nothing new here."  But the performance is a lot higher than people think.

 

Some example datasets where it struggles include the ANLI dataset, and a few reading comprehension datasets.  However, all of it can be explained:  the ANLI dataset is somewhat tricky.  I looked at some of the problems, and on many of them, a reasonable person can reach conclusions at odds with the "gold standard" answers.  A few of them are strange-looking and a little bit semantically ambiguous, given how one interprets what one is reading.  Understanding language is not merely a matter of mapping words to a logical form, and then applying the rules of logic.  There is probabilistic reasoning involved in interpreting what someone means; you have to ask youself, "why would one write that?", and get into their head, to understand what you are reading, sometimes.  For example, if someone said:  Are any of these statements false?:

 

1.  Cocaine is a drug.

2.  Marijuana is a drug.

3.  Beer is a drug.

 

A reasonable person might think, "Beer is a beverage that happens to contain alcohol, but is not considered a drug." 

 

But, then, the grader might say, "WRONG.  Beer is a drug."  It all depends on who the grader is, and the kind of test you are taking.  For example, if you are taking a test in a high school health education class, where the teacher hypes up the danger of drugs and has a cop show up to give a lesson on drug enforcement, then probably they want you to interpret "drug" more broadly; but if you are in some other context, maybe not.

 

It all hinges on what one means by "drug".

 

....

 

Consider the "DROP" dataset:  the model did poor, but not that poor.  However, that dataset has a lot of arithmetic problems on it, and the model has limits to the complexity of the arithmetic it can do without a calculator.  That's because it wasn't taught arithmetic properly, like a human -- it tried to learn it from patterns in text (e.g. "He was born in 2005, and 15 years later, in 2020, he became the youngest athlete to break the world record..." is evidence for 2005+15 = 2020.)

 

On the RACE datasets, although the model is far from state of the art, so are human Mechanical Turk workers!  The humans who took the test turned out to only get about 69% of the harder (high school) RACE problems right and 85% of the easier (middle school) ones right, while GPT-3 got around 47% right on the harder ones and about 58% right on the easier ones.  The state-of-the-art is around 90% on hard and about 93% on easier -- but the state-of-the-art was fine-tuned to do well on the test.  GPT-3 should be compared to human Turkers, not to a model that's had extensive "test prep".  There is still room to improve -- the Turkers are about 25% absolute better; but GPT-3 probably performs only a little below average for high schoolers who take the exam, and about average for middle schoolers who take their portion of the test.

 

Of course on other language tasks, GPT-3 does extremely well.  For example, its question-answering performance is high; its translation performance is high, given that it wasn't explicitly trained to do translation (it's as good as translation programs from a few years ago); its news synthesis is really good (so good that humans struggle to tell it apart from actual news, and this is without any cherry-picking.); and its performance on various verbal commonsense reasoning questions is really high, including ones that were adversarially generated.  And, again, it does all this without fine-tuning.  Out of the box, and given just a few examples to work from, it achieves very high performance.  



#19
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,318 posts
  • LocationNew Orleans, LA

^ The point being that said composers are built off the same architecture as GPT-2, just trained on different things. OpenAI's big project is allegedly going to cross all of the above, plus images/video.


And remember my friend, future events such as these will affect you in the future.


#20
SeedNotYetSprouted

SeedNotYetSprouted

    Member

  • Members
  • PipPipPip
  • 88 posts

 

GPT-2 was already amazing.

 

Amazing, but still nothing more than a text generator, which pulls the "appropriate" words and phrases out its growing database without real understanding of context. Yes, the neural network is "learning" and database is growing, but this is not real AI (at least as I understand it). There are still many years ahead before some GPT-8 will write its first readable procedurally generated book.

 

But with music, things are much better: AI "composers" are already indistinguishable from Beethoven or Mozart.

 

 

You ignored the rest of what he said.







Also tagged with one or more of these keywords: 2020, technology, society, politics

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users