Jump to content

Welcome to FutureTimeline.forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. If you already have an account, login here - otherwise create an account for free today!
Photo

Looks like AI is coming along much faster than people imagined... all that's missing is robots...


  • Please log in to reply
11 replies to this topic

#1
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPipPip
  • 1,961 posts

OpenAI just released an API for developers and academics around GPT-3, and it's amazing! -- that single model enables you to do lots of different tasks.  You basically just provide it with a few examples, and it gets the idea what you want it to do, and then can perform the task for you.  Some of the more incredible examples are here:

 

https://www.futureti...table/?p=282607

 

And here is an example of training the model to map an English description of a task you want it to perform to a line of Linux:

 

Vimeo video

 

And it's still only 2020!  And I think this line of work is going to go much further, when video, audio, and more text are applied -- and maybe also BCI data.  I can already imagine how one might train it to do something as complicated as prove theorems (you'd have to add some iteration, backtracking, and fine-tuning; but I think one could teach it).   And it may even exceed human-level intelligence in certain ways -- you might think that's impossible, given that it's trained on human text, but that isn't so:

 

https://www.futureti...kynet/?p=282105

 

Though, the feed-forward, limited-memory nature of the base model will restrict its ability to outshine humans in all respects.

 

So... we're going to have AIs as powerful as what you see in Hollywood movies, but a lot sooner than people expect.  Even by 2025 it's going to be incredible to see what AI systems can do!   But one thing missing is robots...  Wouldn't it be nice to have a robot like from the film I, Robot, or from the film, Robot and Frank?

 

Roboticists are kind of conservative about estimates on the timeline.  There are a lot of extra hoops that one has to jump through, that aren't true of virtual-environment A.I. -- like:  safety, power efficiency, how to handle physical damage gracefully, the chaos and unpredictability of the physical world, and so on.  At least rudimentary robot bodies (e.g. a Baxter-like robot on wheels; or even something as fancy as Boston Dynamics's robots) will be available to use.  Training them to do things for you is going to take some effort.

 

But, who knows?... Maybe, once again, scale is all you need.  Maybe training very large models on video and physical interaction will result in robot controllers -- somehow -- that are far beyond anything available today.  

 

And maybe it will all come together in a small number of years.  Maybe there will be a robot equivalent of GPT-1, then GPT-2, and so on, in just 3 or 4 years.

 

All those people who made confident predictions that advanced robot AI is 50 years away might find they are very badly wrong, and that by 2029 -- or even 2025 -- there will be general-purpose robot AI systems that quickly learn new tasks from just a small number of human demonstrations.



#2
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,107 posts
  • LocationNew Orleans, LA

There's already a commercially released robot that's in perfect position to show off these sorts of capabilities; it's just been waiting for the right brains

 

It's cheap enough at the moment, at least.

 

 

By 2025, we could definitely see this be put into something like the UBTech Walker (which itself is like a commercial ASIMO), or perhaps an even better bot than that.


And remember my friend, future events such as these will affect you in the future.


#3
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,107 posts
  • LocationNew Orleans, LA

And definitely, by 2029, Corridor's videos about "Atlas going rogue" could probably be done with Atlas itself with very little pre-scripting.


And remember my friend, future events such as these will affect you in the future.


#4
tomasth

tomasth

    Member

  • Members
  • PipPipPipPipPip
  • 294 posts

"quickly learn new tasks from just a small number of human demonstrations"

And figure out what action to do from explanation hints and in some cases on their own from procesing the context ?



#5
Cyber_Rebel

Cyber_Rebel

    Member

  • Members
  • PipPipPipPipPip
  • 397 posts
  • LocationNew York

I feel like robotic/autonomous police are going to be a controversial inevitability, given the current day issues. Silicon Valley & Dubai actually had already dabbled in primitive forms of this. 

Still, I never appreciated the want/need for a utility bot until this pandemic hit. Here's to hoping that the "robot revolution" may be the highlight for the later halve of this decade. I hope they'll eventually become as ubiquitous as smartphones are. 

 



#6
scientia

scientia

    Member

  • Members
  • PipPip
  • 10 posts

A lot of times, people get upset when I post. So, if you find this upsetting, just let me know and I'll leave the conversation to you.

 

OpenAI just released an API for developers and academics around GPT-3, and it's amazing! -- that single model enables you to do lots of different tasks.  You basically just provide it with a few examples, and it gets the idea what you want it to do, and then can perform the task for you.  Some of the more incredible examples are here:

I'm not seeing anything that would be useful in an AGI system.

 

And it's still only 2020!  And I think this line of work is going to go much further, when video, audio, and more text are applied -- and maybe also BCI data.  I can already imagine how one might train it to do something as complicated as prove theorems (you'd have to add some iteration, backtracking, and fine-tuning; but I think one could teach it).   And it may even exceed human-level intelligence in certain ways --

 

Maybe once the amount of data and depth of the network gets beyond a certain point, the system starts being able to go significantly beyond human intelligence (within the confines of the feedforward compute limits).

As far as I can tell, this system has no possibility of ever being human-level.

 

So... we're going to have AIs as powerful as what you see in Hollywood movies, but a lot sooner than people expect.  Even by 2025 it's going to be incredible to see what AI systems can do!   But one thing missing is robots...  Wouldn't it be nice to have a robot like from the film I, Robot, or from the film, Robot and Frank?

So, what if I told you that I worked on the basis for the control system for robots like you see in I Robot back in 2017? And what if I told you that Asimov's Three Laws can't be programmed into AGI system? I haven't found anything yet that would prevent human-level intelligence, but the control system wouldn't be small enough to fit inside a human-sized robot sooner than 2045. Once you have the controller, there are still three issues for robots that I haven't seen solutions to.

  1. HIgh density power source
  2. Durable covering
  3. Speed versus torque

I'm not an engineer so perhaps these have solutions I'm not aware of.
 

Cyber_Rebel

I hope they'll eventually become as ubiquitous as smartphones are.

I'm still wondering what people plan to do with a robot. They probably won't be more reliable than a human worker. And the cost would probably be higher. The last survey I did was over $250,000 for a minimal unit.



#7
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,107 posts
  • LocationNew Orleans, LA

A lot of times, people get upset when I post. So, if you find this upsetting, just let me know and I'll leave the conversation to you.

 

OpenAI just released an API for developers and academics around GPT-3, and it's amazing! -- that single model enables you to do lots of different tasks.  You basically just provide it with a few examples, and it gets the idea what you want it to do, and then can perform the task for you.  Some of the more incredible examples are here:

I'm not seeing anything that would be useful in an AGI system.

Because GPT-3 is not an AGI nor has the architecture needed for AGI.

Don't misunderstand what people are writing here; no one serious is under the impression that it's GPT-3 that will lead to AGI. It's something else entirely.

 

 

What GPT-3 does qualify as is something different. Still generalized, but not "general AI."


And remember my friend, future events such as these will affect you in the future.


#8
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPipPip
  • 1,961 posts

So, what if I told you that I worked on the basis for the control system for robots like you see in
I Robot back in 2017?

 
Well, as a STEM professor myself (not in robotics; full-professor; tier-1 research university), I am not the least bit intimidated by people with technical talent:
 
https://www.futureti...ow-automatable/
 
There's even the chance you were one of my students, at some time in the past!  (I am not young.)
 
(And, yes, I realize those problems are a bit-out-reach -- mainly due to the length -- at the moment; but not too far out of reach.)
 

As far as I can tell, this system has no possibility of ever being human-level.

Define what you mean by "human level", and be precise. Because, I have little doubt that you have learned nowhere near as many basic facts as the system can pull up. In that case, it is "superhuman". Not only has it memorized facts, but it can also blend them together. Even if you had access to the internet, to look up facts, there might be some very simple tests / tasks that it could beat you at.

....

I thought it might be worth mentioning what OpenAI might work on next:  it looks like they want to continually improve their API, which can already do some amazing things.  However, it has been reported -- or suggested, in a Tech Review article -- that they also want to train something like a GPT-3, but multimodal (GPT-4?  Something else?).  In other words, it would not only use text from which to build its internal representations of knowledge, but also audio, video, and images.  
 
Let's say that images can be compressed down into 10,000 bytes each.  You can do fairly well with that level of compression -- hit major details.  
 
So, then, if they train it with 100 million images, that adds up to 1 terabyte.  
 
And now, let's say audio clips are compressed to maybe 10,000 to 100,000 bytes, depending on the length, using e.g. neural net-based compresson.
 
Video is harder to compress; but it can be done.
 
Ok, so, maybe they train a system with several terabytes of images, audio, video.  How would they do it?  Well, they probably would have to alter the architecture.  Gwern suggests they may add new attention mechanisms; and that's probably the least of the modifications.   And maybe they will make the context window a lot longer, so that it's over 100,000 tokens long, instead of 2,048.  
 
It might make sense to mix the modalities together, as they appear in the real world.  For example, when someone writes a post to Reddit, in the middle of their post, they might link to a jpeg image.  One way to represent that post is as a combination of text, with special tokens in the middle to represent the compressed form of the image.   That might not actually work -- but there's a chance that it would.
 
Do the same for audio and video, too.
 
Now, train a giant model with many more parameters than the current one they trained.  Maybe this takes $20 million... or even $50 million... or more.
 
What might one do with it?...  
 
Here's some suggestions -- one, unified system to do them all:
 
1.  Let's say you are business owner, and you have some old records, in different formats, that you scanned into your phone over the years, and onto a computer.  You've got thousands and thousands of jpeg files, with digitized documents.  They're not in uniform font; some have little logos embedded; and so on.  You can't pay a company to automatically map it to text for you.  But, maybe you could feed into some next-gen version of GPT-3, and say, "Make a list of the names of the customers in all these records," and then list out the records (as compressed image files).  Maybe you have to give it some examples, first, to prime it, so it knows what you want.
 
In the process of learning to predict tokens (including in images), maybe it learns good representations of the content of images -- to such a degree that it learns Optical Character Recognition implicitly.  In fact, you can even write characters up the side of the page, rotate them, in different fonts, and it still works. You can put distractors in the image, include little photos; and it will still work.
 
2.  Let's say you have some bar charts and pie charts and other kinds of charts, and just for your own amusement (not for any serious application), you ask the computer to map them to spreadsheets.  So, it has to recognize the number scale being used (by reading the numbers on the chart), recognize that the height of the bars says how large the numbers are, read off the names of the items at the bottom of the chart, and so on; and it has to tolerate lots of different kinds of charts.  And it has to know how to map that to a CSV file, for example.  It's a lot of work! -- but... there are going to be millions of graphs like that in the training data, that the system will have learned useful features from.
 
3.  Maybe you want to test its ability to interpret diagrams?  So... you take a photo of a circuit diagram, and write to API, "What is the resistance between terminal A and terminal B?"  And then you add the photo.  Now, the model will have seen many thousands of diagrams like that, with accompanying text describing the circuits; so it will have representations for that, too.  So, it may look at the image, recognize the squiggly resistor symbol, see the numbers written next to it, and then report that.  It' possible that it will even be able to do some rudimentary Ohm's Law calculations, and answer questions that must be derived from the image.
 
4.  Let's say you give it a couple examples of audio clips (that include singing and multiple instruments), followed by a short MIDI file representation of just the core melody in a single "voice" (no harmony).  Then you give it one more audio clip, and ask it to complete... And it will output the MIDI for the core melody.
 
5.  Let's say you have a couple short video clips of people preparing some basic meals.  There are several different ingredients involved; and several steps in the cooking process.  Just for your own amusement (not that you actually want to use it for anything), you show the system some examples of "here is the video" and "here is the recipe".  Then, you show it one more video, and it completes it... outputting a plausible recipe.  It figures out what you want it to do.
 
Now, if you have a system that can do those things -- and those are only a tiny snapshot of its capability -- just think how much more you could do with OpenAI's API when it gets that capable...



#9
scientia

scientia

    Member

  • Members
  • PipPip
  • 10 posts

Because GPT-3 is not an AGI nor has the architecture needed for AGI.

Don't misunderstand what people are writing here; no one serious is under the impression that it's GPT-3 that will lead to AGI. It's something else entirely.

You see GPT-3 as a type of AXI.

 


What GPT-3 does qualify as is something different. Still generalized, but not "general AI."

To be honest, I'm not sure that you understand what general means. There is no such thing as a general AI. This would be a violation of information theory. An actual AGI is outside of computational theory. My impression is that your AXI is entirely computational, so it would be a type of AI. So, I'm not seeing how you believe it would be more general. GPT-3 is a larger version of GPT-2. It isn't smarter or more general. So, what would be the basis for a more generalized AI?



#10
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 22,107 posts
  • LocationNew Orleans, LA

Actually, let me rephrase this: GPT-3 appears to operate on two layers of functionality, one being deliberately designed and the other being emergent (not to be confused with deep neural network layers). The underlying layer is based on language modeling, which is a fairly simple function that is not natively capable of general intelligence or anything of that nature. 
 
What gives it generalization capabilities is how complex its training data has becoming; building off of text and language modeling, it is capable of branching out into different tasks, several of which it was not initially trained to do. So basically the language model is like the seed or roots and the other capabilities are like the branches and leaves.

 
These tasks would normally require hardcoded individual networks to accomplish. Indeed, in some cases, it'd be easier to just get a single AI to do any number of those individual tasks, but that's not the point. Showing off GPT-3's ability to do multiple tasks is indeed quite the point of the demonstration.
 

So, I'm not seeing how you believe it would be more general. GPT-3 is a larger version of GPT-2. It isn't smarter or more general. So, what would be the basis for a more generalized AI?

For this, ask Starspawn0.


And remember my friend, future events such as these will affect you in the future.


#11
scientia

scientia

    Member

  • Members
  • PipPip
  • 10 posts

So, with some work, GPT-3 might be able to replace Alexa? Is there anything else you can do with it?



#12
Metalane

Metalane

    Member

  • Members
  • PipPip
  • 34 posts

So, with some work, GPT-3 might be able to replace Alexa? Is there anything else you can do with it?

GPT-3 *itself* would't replace a voice assistance, but a software akin to it would. 






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users