Jump to content

Welcome to FutureTimeline.forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. If you already have an account, login here - otherwise create an account for free today!
Photo

True Artificial Intelligence Could Be Closer Than We Think, Via Brain-Computer Interfaces + Deep Learning

AI BCIs Brain-Computer Interface Artificial Intelligence

  • Please log in to reply
112 replies to this topic

#41
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 20,705 posts
  • LocationNew Orleans, LA

Just curious: why are you sure machines are not (relatively) close to passing the Turing Test? What particular capabilities do they lack?


And remember my friend, future events such as these will affect you in the future.


#42
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
The main problem is that set of possible directions a conversation can veer is so large that it's hard to program machines for every occasion. Language Understanding (accurate slot-filling) is also difficult; but handling all possible conversations is harder.

People try to give computers some logical reasoning, planning, and other abilities to account for all the possible twists and turns in a conversation -- but these methods are too "brittle" to handle general conversations. And people aren't particularly logical in conversation -- maybe more complex logics can handle "conversational logic", though.

Brain-based AI as I described in this posting

https://www.futureti...might-be-built/

is the kind of approach I could see might lead to a Turing Test-passing AI.

Another approach is to fake it, and hope to get it to work a large percent of the time, convincing most people. This is the following one some of the teams at the Alexa Prize competition are pursing:

* Create a giant diagram (graph or network) of conversation/dialog states. A "state" here represents a possible turn in a conversation; for example, it might represent, "conversation turn about books, asking about favorite author". Conversations are then to be modeled as transitions between states in the graph.

* Make the graph very, very large -- tens of thousands, or even hundreds of thousands of states, representing practically any kind of conversation state you can imagine. If the graph is large enough, even if the states don't exactly match the present conversation, there will be one that comes "close enough".

* Each state/node in the diagram has an associated set of "slots" that get filled when in that state in a conversation. For example, if the state is about "books", one of the slots will be "author of the book"; another might be, "protagonist or main character".

* When transitioning from one state to another, an associated "script" will be printed, that generates the response. For example, one of these scripts might be, "The author of <book> is <book_author>."

* Or, perhaps a module is called to act on the information in the slots (e.g. to do reasoning); followed by the print-out of a script.

* Now, one needs something called a "dialog management" system to do "dialog state tracking" -- basically, figuring out what state you're in at each point in a conversation.

* One also needs a "Language Understanding" system to figure out how to extract "slot values" from a turn in a conversation. For example, if you say, "The author of Dune was Frank Herbert", a language understanding module would set the slots "author" and "book" to "Frank Herbert" and "Dune".

* And those "modules" I alluded to will need to be built. These might do things like solve math problems, address basic reading comprehension questions, look up facts, and so on.



That's the standard way to build virtual assistants and bots. And a sufficiently complex one, with a sufficiently large dialog state graph, sufficiently good language understanding, and sufficiently large number of high-performing modules, should be able to fake a natural conversation pretty well.

The hope would be to use Machine Learning somehow to learn a large dialog state graph, along with slots and scripts, so that extremely large graphs can be constructed. Maybe millions of states can be induced automatically, without hand-crafting.
  • Yuli Ban and Alislaws like this

#43
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
A Twitter posting by a visible young neuroscientist:

https://mobile.twitt...229354283868160

imho new latent state models of behavior + neurons are THE most interesting thing happening in neuroscience right now.


He links to two new papers in that thread:

https://www.biorxiv....18/07/25/376830

https://www.nature.c...467-018-04723-6

This kind of thing is in the same circle of ideas as using Deep Learning to predict neural populations; but the methods are more basic.
  • Yuli Ban likes this

#44
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
Leslie Valiant (in case you don't know who he is, he is one of the "founding fathers" of modern machine learning, having introduced the PAC-learning framework. He is a professor at Harvard.) just posted a paper to the arxives on "system-level" primitives that underpin cognition:

https://arxiv.org/abs/1807.10374

I haven't read it yet, but from what I've seen thus far, it seems he thinks that the "neuron-level" and lower are not the right things to look at:
 

The hypothesis considered here is that cognition is based on a small set of systems-level computational primitives that are defined at a level higher than single neurons. It is pointed out that for one such set of primitives, whose quantitative effectiveness has been demonstrated by analysis and computer simulation, emerging technologies for stimulation and recording are making it possible to test directly whether cortex is capable of performing them.


Probably, these primitives are expressed at the population level (many, many neurons, not just a small number), and "emerging technologies" like BCIs will be able to detect them.
  • Yuli Ban and Alislaws like this

#45
Alislaws

Alislaws

    Democratic Socialist Materialist

  • Members
  • PipPipPipPipPipPipPipPip
  • 2,027 posts
  • LocationLondon

Leslie Valiant (in case you don't know who he is, he is one of the "founding fathers" of modern machine learning, having introduced the PAC-learning framework. He is a professor at Harvard.) just posted a paper to the arxives on "system-level" primitives that underpin cognition:

https://arxiv.org/abs/1807.10374

I haven't read it yet, but from what I've seen thus far, it seems he thinks that the "neuron-level" and lower are not the right things to look at:
 

The hypothesis considered here is that cognition is based on a small set of systems-level computational primitives that are defined at a level higher than single neurons. It is pointed out that for one such set of primitives, whose quantitative effectiveness has been demonstrated by analysis and computer simulation, emerging technologies for stimulation and recording are making it possible to test directly whether cortex is capable of performing them.


Probably, these primitives are expressed at the population level (many, many neurons, not just a small number), and "emerging technologies" like BCIs will be able to detect them.

I don't understand most of it, but this is my favourite thread!


  • starspawn0 likes this

#46
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
It looks like a team has built an FNIRS-DOT (Functional Near-Infrared Spectroscopy - Diffuse Optical Tomography) scanning device using a special high-intensity, ultra-high-quality CCD (Charge Coupled Device) camera, that is able to scan the entire brain of a young child with millimeter resoluiton and temporal resolution at least as good as fMRI (probably even better than fMRI):

https://www.spiedigi...2.2293516.short

Abstract:

Although fMRI technique can be used to scan the whole brain to investigate the brain functional activities in children by using the blood oxygenation level dependent (BOLD) MRI signal, patients often require sedation, which prevents investigation of brain function during the actual disordered movements. Functional near infrared spectroscopy and diffuse optical tomography (fNIRS-DOT) allows a similar assessment to the BOLD MRI signal. However, in a realistic situation there will be a skull, blood vessels, cerebrospinal fluid and a significant variation of distance of the deep hemovascular target from the tissue surface. Transmitted diffuse near-infrared (NIR) light is significantly scattered and attenuated, making it difficult to be detected with sufficient signal-to-noise ratio (SNR) using conventional detectors such as avalanche photodiodes or photomultiplier tubes. In this study, we present an fNIRS-DOT imaging system by use of military based intensification with charge-coupled device (CCD) technology to acquire the transmitted weak diffuse NIR signals with high sensitivity that allows whole brain imaging and the device can be mounted in a comfortable cap on an awake child. The millimeter level spatial resolution of transillumination tomographic reconstructions was demonstrated in studies of phantoms that approximate the size of a child’s brain. In addition, we have been able to obtain preliminary transcranial proof-of-concept data with the reasonable SNR in a 7 month old healthy baby. Further tomographic studies will allow the assessment of the brain network dysfunction in awake children suffering from brain diseases such as brain tumor.


What's the catch? Well, for one thing, the cap has a bunch of wires coming out of it; but maybe something like it could be built as a self-contained system worn around the head.

Also, it has so far been tested in children; but I see no reason it won't work in adults, with a small loss in resolution in order to look deeper into the brain.

It uses several different wavelengths, but should still be able to at least pick out the BOLD signal, making it at least comparable to fMRI at the same spatial and temporal resolution -- and the temporal resolution can be improved, probably.

So, it looks like it has most of the requirements for lots of amazing AI and BCI applications. Just needs some shrinking (self-contained, wearable) and development on the application side.
  • Yuli Ban likes this

#47
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
David Sussillo wrote a post for the Simons Foundation website about "neuroscience is about to get bonkers", that is more in-depth than his recent tweet storm I posted earlier:

https://www.simonsfo...to-get-bonkers/
 

I think the systems neuroscience community should think of ANNs trained to model tasks as a kind of synthetic model organism.


That will be an unavoidable conclusion as time goes on -- as these systems will behave in more animal-like ways. But probably not if they are just trained on tasks; rather, it will happen when they do the "model the data" approach he describes.
 

Model the data: Optimize a neural network that generates simultaneously recorded neural data. Modeling the data is useful because it provides a concrete network that may give insight into how the neural data is generated.


In this new posting, he advocates not only trying to model individual brain regions, but to combine models from different brain regions, and possibly model the whole brain:
 

Do what we do now, but record from more neurons across more brain regions. Researchers are already using technologies such as Neuropixels to measure brain-wide responses to simple, well-studied behaviors. These types of studies will give insight into how neurons in different brain regions work together to perform computations.


At the same time, he advocates much more complicated behavioral tasks to train and probe models:
 

Do what we do now, but make the tasks we study more complex. To capture the brain’s full computational repertoire, we may need to use much more complex behavioral tasks. Of course, more complex tasks will also produce more complex data, which will in turn be more challenging to interpret.


And he thinks Deep Learning will be a major key to making all these models work, and spends a few paragraphs explaining how.

He's basically describing the predictions I have made over the past several years, except he is doing it for animals with neuropixel and Utah array probes. Here is where I differ:

* It will be great when many neuroscience labs share data and do all these experiments. However, this pales in comparison to the amount of data that will be generated by inexpensive, high-quality, non-invasive BCIs in a couple years (starting in the next year).

* I envision something like a "Maker revolution" where people use BCIs to control drones; do typing-by-thought; control robots; determine which of 100 images someone is thinking of -- and lots more. A lot of it will be very kitschy; but it will draw lots of people and money to the budding BCI industry. There will be a lot of experimentation, and plans for neural encoding and decoding models (software and blog postings on the optimal parameters) will be widely shared.

* During this explosion of activity and dissemination of machine learning models to work with the data, some number of people will try to build AI models from the brain data. These will probably appear in the academic literature first, and will filter out to the Maker community. Some of these might not be very good; others will frighten people by how complex the tasks they are able to perform -- for example, interpreting video or critiquing blocks of text.

* It will be much harder for people to deny that the AI systems are doing something analogous to what the human brain does. It will be harder to say, "But it doesn't really UNDERSTAND what it's reading." If it's built from the human brain, on some level it is "understanding" like a human.

* Comparisons will be made to zombies and Frankenstein. I expect we may see some religious nuts attack scientists and companies pursuing this line of work.

* Once some really good "zombie AGIs" are built with brain data and Machine Learning, the economic impacts will be sweeping and rapid. We'll have socialbots that can pass as human a large percent of the time. There will be a dark side, as they take to social media to influence voting patterns. Some may hack into computer systems as well as trained hackers.

* Perhaps the greatest potential threat is if someone finds a way to make the AIs smarter and meaner. I can imagine several ways that may happen. It may, ultimately, not be all that hard -- some Makers may work out how to do it by using Neuroevolution; and they will share what they've learned.
  • Casey and Yuli Ban like this

#48
tomasth

tomasth

    Member

  • Members
  • PipPipPipPipPip
  • 239 posts

Its good that its crud.

 

 Deep Learning will be usefull even if its not how the brain does it and whatrthey do it very approximate functionally.

 

Its good that people say "it doesn't really UNDERSTAND what it's reading" , they will accept it nore and see it as only a tool and and aid.

 

"religious nuts attack" more religious people have no issue with this anymore then with smarphone or VR , makeing it more acceptable to them (only a tool an aid) is good.

 

The way of VR and phone apps , should popularise the hardware. After the ball it rolling , compenies can invest in the higher res and in more biological functionally relevent models of neural net.

 

Deep Learning With the coming of crud low power/price neuromorphic chips , maker can have more divers uses then with cloud dependency.



#49
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
Another observation about how to make the rough emulations smarter, and more robust: use ensembles.

Perhaps you have heard of the famous experiment of Galton about the "weight of an ox":

https://en.wikipedia.../Francis_Galton
 

In 1906, visiting a livestock fair, he stumbled upon an intriguing contest. An ox was on display, and the villagers were invited to guess the animal's weight after it was slaughtered and dressed. Nearly 800 participated, and Galton was able to study their individual entries after the event. Galton stated that "the middlemost estimate expresses the vox populi, every other estimate being condemned as too low or too high by a majority of the voters", and reported this value (the median, in terminology he himself had introduced, but chose not to use on this occasion) as 1,207 pounds. To his surprise, this was within 0.8% of the weight measured by the judges. Soon afterwards, in response to an enquiry, he reported the mean of the guesses as 1,197 pounds, but did not comment on its improved accuracy. Recent archival research has found some slips in transmitting Galton's calculations to the original article in Nature: the median was actually 1,208 pounds, and the dressed weight of the ox 1,197 pounds, so the mean estimate had zero error. James Surowiecki uses this weight-judging competition as his opening example: had he known the true result, his conclusion on the wisdom of the crowd would no doubt have been more strongly expressed.


So, if we had rough emulations from multiple different brains -- if we could choose them independently and uniformly at random -- we could average them, or compute a weighted average, when trying to reach conclusions such as determining quantities from visual input (e.g. counting the number of people in a scene; estimating the height of a person; weighing an ox).

But this average wouldn't necessarily feed back in to affect later judgments from each of the brains. I see two options here:

* We could try to model all the brains jointly (i.e. build a "joint encoding model"), sharing parameters in predicting all the different brain responses. This might increase stability. For example, if one encoding model for one brain gets a little off-track, the other models would contribute their predictions to knock it back into a stable configuration. The joint model would make the individual models even more stable

* We could try to model a weighted average of all the brains (suitably aligned) while passively observing stimuli. It's possible that the weighted average of several brains actually has an "identity". It wouldn't have long-term memory, other than shared knowledge of the world, as the effects of individual long-term memories would get washed-out in the average. On problems like weighing an ox, and any other situation where the "wisdom of the crowd" reigns supreme, it would be scary-accurate. It would also probably have good (or at least human average) language understanding and commonsense reasoning, as those things are similar and shared across individuals. And, it's moral deductions would probably tend toward what most would describe as "good", being the average of human judgment. Lastly, the signal to be modeled would have much better signal-to-noise ratio, since averaging tends to remove noise (regression to the mean, Law of Large Numbers, Central Limit Theorem).

In fact, on the last point, some of the examples I listed earlier in this tread worked by modeling the average behavior of many brains. For example, I recall that this is what the "Deep Learning the human mind" paper did -- it learned a model of EEG readings of several brains averaged together, because that happens to have a much better signal-to-noise ratio -- the noise in EEG recordings gets averaged away.

#50
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
Yuli-Ban asked me to post an "update" here, so I thought I would drop by for one posting.
 
There has been a lot of work related to the OP over the past several months.  I can't list all of it, so will only list a few highlights.
 
First, and most important, there is a MASSIVE new project, with approximately $5 million in funding (the figure was given in Canadian dollars as $6.3 million), to collect 900 hours of fMRI+MEG data from each of 4 individuals -- about 3,600 hours total.  They refer to this as "12 subjects"; but it's actually 4 subjects where each contribute 300 hours on language tasks, 300 hours on video tasks, and 300 hours on memory tasks, if I understand correctly.  That's an UNBELIEVABLE amount of data from single subjects. A Twitter thread about it can be found here:
 
https://mobile.twitt...740262401249280
 
And a video (in French) about the project can be found here:
 
YouTube Video Le projet de recherche Courtois Neuromod
 
Depending on the kind of language and video tasks they record, this could have a huge impact on AI.  e.g. If they have subjects read a large amount of text while doing eye-tracking, I could see this leading to very large improvements in Natural Language Understanding; and if the videos are of a broad enough category of types (e.g. news, movies, lectures, short clips, games), then the video+brain+physiological data could vastly expand video understanding, as I have discussed.  I don't speak French, but I think in the video above one of things the speaker suggested was that they plan to use brain data as people play games to build a brain imitator model (using Deep Learning) to play the games like a human would.  
 
So, in 5 years we might get our first taste of TRUE AI.  It would not be full AGI; but closer to the low-grade "Zombie AGI" I've talked about before -- still, it will be AI of a very different kind than we've seen today; and very different from all the other types people have attempted to build using elaborate "cognitive architectures".
 
This is, in fact, one of several projects that are in the works; but will involve more brain data per subject than any I've heard about, by a mile.  And if these projects are successful, it will snowball -- more and more groups will collect larger and larger amounts of data to train models.  
 
In the Twitter thread above, it specifies the goals (in English):
 

The #1 aim of the project is to use the neuroimaging data to constrain the training of deep neural networks able to transfer over different tasks, in a variety of cognitive contexts (in particular vision and language). Secondary aim is to build new models of brain networks.


This #1 aim sounds like they want to figure out what kinds of Deep Learning architectures would be best to use to generalize in the way the brain does. This will probably also involve fitting models to the data, to see which does a better job imitating the brain.

....

Another thing worth mentioning is that a guy named Gwern, who seems to be a Less Wrong and Slate Star Codex person (I don't follow those communities; he is one of several people from those groups on my forum), wrote a nice summary of the "brain imitation" approach I described:

https://www.reddit.c...ation_learning/

He also mentioned this to some people in the AI safety community on Twitter:

https://mobile.twitt...729464406949890

So it is now something these people are aware of (there is also a guy with OpenAI connections who joined my forum, who now is aware of the approach these groups are taking). I think in the future they will be investing more time looking into this way of building AI, as the data starts to roll in.

....

One final thing worth mentioning is this work of André Ofner and Sebastian Stober:

https://arxiv.org/abs/1810.02647

They aim to use EEG data to improve AI, with a goal of possibly eventually producing AGI:
 

Within the context of biohybrid systems, it has been suggested that the process of "seeding" an artificial agent with data from humans might provide a basis for a high level of cognitive capacity. This is in line with the previously results from studies that have used projections of stimuli into associated brain states as a means to enhance the capacity of deep learning models. While this happened in a uni-directional process, we suggest that in a closed loop this effect might increase.


While this is almost certainly not going to happen with EEG (unless one somehow can harvest millions of brains), as EEG is such a poor BCI / brain scanner, we might see their method succeed using next-gen BCIs, such as a later-generation of Openwater's device (alpha kits are said to be out in summer 2019).

The second author, Stober, by the way, gave a talk recently at the Montreal MILA Main 2018 conference:

https://mila.quebec/en/2018/11/31415/

The above work (Hybrid Active Inference) was one of the topics he discussed.
  • Casey, Yuli Ban and Alislaws like this

#51
Yuli Ban

Yuli Ban

    Born Again Singularitarian

  • Moderators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 20,705 posts
  • LocationNew Orleans, LA

I'll be here to forward some more comments from Starspawn0. My particular favorite at the moment:
starspawn0

I have a reasonable confidence that that many hours from a single individual would be enough for video understanding and language understanding that would make a mockery of the absolute best work Google and other groups have produced to date. It would be like comparing speech recognition circa 2000 to speech recognition circa December 30, 2018 -- there is no comparison.


  • Casey and starspawn0 like this

And remember my friend, future events such as these will affect you in the future.


#52
Casey

Casey

    Member

  • Members
  • PipPipPipPipPipPip
  • 676 posts

In other areas of neuroscience, if I'm reading this article correctly, improvements in algorithms might be able to simulate the brain using 1 exaflop computers. Not sure if I'm understanding that right, but I really do believe that the brain will transition from being an infinitely complex mystery to something we understand very well as the decade goes along, and that by the latter half of the '20s the brain will basically become our bitch between vastly more powerful supercomputers along with the BCI revolution. I agree with TranscendingGod's user title: I think the 2020s will be a phenomenal decade and one of the very most important in human history.

 

I've always thought to myself that, on my deathbed, I would like to laugh at all the techno-skeptics that were always so smugly confident that the world would never get better and no field of technology would make any kind of meaningful advances ever - the people that always made me stressed and nervous when I was young during the olden stone ages of the early 21st century. I still feel that way, of course, but I'm glad that the '20s should give me some early bragging rights as well.


  • Yuli Ban and starspawn0 like this

#53
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts

In other areas of neuroscience, if I'm reading this article correctly, improvements in algorithms might be able to simulate the brain using 1 exaflop computers.


I don't think the brain emulation approaches we hear about are going to pan out anytime in the near-future. The brain is just too complicated to emulate using our present technology. BUT, we can still imitate it, which is what this thread is about. (Gwern did a nice job highlighting the differences between "emulation" and "imitation" in his posting, by the way.)

....

My main reason for showing up again was to point out something about the above video:

YouTube Video Le projet de recherche Courtois Neuromod

As I said, it's in French, and you may not be able to understand French. However, YouTube allows you to translate, though the translation is much poorer than Google Translate applied to a text document. Here are the steps:

* Click the gear icon as you play the video.

* Then, set subtitles to French.

* Then, click auto-translate.

* Set target language to English. You'll have to wait about 3 seconds are it loads.

* Finally, you might click the gear icon again, and slow the video down to 25% normal speed, as the subtitles / captions jump around in a distracting way.


I recommend watching the first 2 minutes or so. That is when Pierre Bellec, a main scientist on the project, discusses how one of the things they want to do is to use brain and behavior data to build models that can imitate human gameplay, such that if you look at the internal states and dynamics of the model, it will match what you see in the brain of a human player (and it should have strong transfer, so that you could apply it to practically any game, Atari, Nintendo, modern first person action shooters, adventure games, car driving in diverse environments, etc.; and using some Reinforcement Learning, Deep Learning, and/or Neuroevolution, the performance could be pushed to super-human levels on any game). That's exactly what brain imitation is all about. He's basically implementing the project I described, and using the very large amounts of data I fantasized about being available in the near-future. But he is getting that data from fMRI, not BCIs.

There is one slight cause for concern, though: fMRI has a temporal resolution of about 1 to 3 seconds, and the BCIs I am hoping to see should have a resolution of about 100 milliseconds. That could, in principle, greatly weaken the usefulness of the data. However, he is recording so much of it, that it should be possible to use Deep Learning to "fill in the gaps". So, I think it will be a big success -- something like ImageNet, but for higher cognition. It will greatly accelerate the pace of AI development.
  • Casey, Yuli Ban and Alislaws like this

#54
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
I found this interesting: Pierre Bellec ran a contest to design NeuroMod's logo (I guess when you have a project their size, you need a good logo!). There are some pretty good top designs, that are just as good as the one that won:

https://99designs.co...oscience-830448

He also gave another description of the project (the more descriptions, the better):
 

Tell us a bit about who you are and the people you reach

NeuroMod stands for "neuronal modelling". It is the short name of the Courtois Project for neuronal modelling, a private-based science initiative, which aims at building artificial intelligence systems that replicate the brain activity measured in individual, human subjects. The project is first a research enterprise, pushing the envelope of artificial intelligence models as well as neuroimaging acquisition & processing. It will alos be a resource: all data and models will be shared freely with both the AI and Neuroscience communities.


It would be good to hear more about the individual systems they hope to build with the brain data -- e.g. whether they want to build a "video understanding module" to answer questions about what happens in a video.

....

Another thing worth mentioning: Daniela Rus was interviewed by Martin Ford for his new book "Architects of Intelligence". You may recall that she is the director of MIT's CSAIL, their Computer Science and Artificial Intelligence Lab. Anyways, in the middle of the interview, while talking about AGI and the future of understanding human intelligence, ford asks her if she sees any "extraordinary breakthrough" on the horizon that really moves the field along, and she said, basically, BCIs:
 

Martin Ford: Is it possible there might be an extraordinary breakthrough that really moves things along?

Daniela Rus: That's possible. In our lab, we're very interested in figuring out whether we can make robots that will adapt to people. We started looking at whether we can detect and classify brain activity, which is a challenging problem.

....

With the external sensors we have today, which are called EEG caps, we are fairly reliably able to detect the "you are wrong" signal.

....

What's interesting, though, is that these EEG caps are made up of 48 electrodes placed on your head -- it's a very sparse, mechanical setup that reminds you of when computers were made up of levers. On the other hand, we have the ability to do invasive procedures to tap into neurons at the level of neural cell, so you could actually stick probes into the human brain, and you could detect neural-level activity very precisely. That's a big gap between what we can do externally and what we can do invasively, and I wonder whether at some point we will have some kind of Moore's law improvement on sensing brain activity at a much higher resolution.


So there you have it, from MIT CSAIL's director.

....

And... there's a lot more stuff out there related to this line of work. I've only scratched the surface. The most exciting stuff in the near-term, however, is NeuroMod. This year they hope to release the first batch of their data -- that, alone, could spark advances in AI; but probably it will have to wait for another couple more years.

It's possible, though, that they will use that data to build their human-like game playing system this year. I'd say 1 year's worth of data ought to about do it for that project, for an assortment of different types of games; the fancier games, though, will probably require modelling more of the brain, and will take more data -- that's just a guess.

Even one success from a project like that could create a mad-dash to collect more data. Everybody and their uncle will be collecting brain data if they can use it to do something shocking. Big companies may even buy an assortment of fMRI and MEG scanners to collect reams of data -- they won't wait for those BCIs to come out. We could be about to witness a runaway growth phase in AI development.
  • Casey, Yuli Ban and Alislaws like this

#55
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
Four more things to add to this thread:

First up is a podcast interview with Joshua Glaser, a neuroscientist who works in Konrad Kording's lab (as I recall):

https://braininspired.co/podcast/8/

Glaser has done some work on modelling brains using Deep Learning. During this interview he was asked by Paul Middlebrooks how far away we are from modelling the whole brain:
 

Paul Middlebrooks (PM): How far off are we from modeling the whole brain? I mean, you know, you keep putting these networks together, and joining them, and explaining the movement and explaining the sensory, and we're getting closer and closer to these watershed prefrontal cortical areas and things like that. How far off do you think we are?

Joshua Glaser (JG): I'd still say pretty far off.

PM: Take the over-under on let's say 20 years.

JG: I think saying "modeling the brain" is a little too vague to put a number on... or even a success criterion on it. So...

PM: Well, I guess what I really mean is... let's say the criterion is sufficient to use feedforward convolutional neural nets to model sensory areas, right? And then, you have recurrent neural nets to model movement areas, and things like hippocampal place cells. So, we're getting to the point where these models are spreading throughout the brain -- not that we even understand enough about brain areas, in general; I mean, these are really well-understood brain areas we're talking about, studied for a long time, so... I guess there's a lot more neuroscience to do as well. But, I'm just kind of envisioning these pieces falling into place, and at some point we're going to connect them together, and it'll be a massive failure for *acting* like a real brain, I think; but I'm... uh... picturing it. I'm wondering when all those pieces will be available to plug in together...

JG: For that question, I'd say we're probably not super-far off. Again, I don't have an exact time estimate; but from when we will be able to take a lot of neural nets that have brain-like properties, let's say, and put them together to form some larger system that can do a lot of what humans can do, and have a lot of the same properties as the brian... I'd say for something like that, just to guess, maybe 10 years.

PM: Oh, wow! Alright! I like your attitude there.

JG: That being said, I think that system is very far from what the brain actually is; and, just because you train a neural network and activity emerges that looks very similar to activity in the brain, does not necessarily mean that this is what the brain is doing. It can just be that, essentially, any type of computing system, even if the brain is doing it completely differently from a neural net, will solve this type of computational problem of how to classify an image [and process language, and understand video and audio], it will do it in a similar way. And so, ultimately what I can see originally predicted in 10 years may still be very far from actually understanding how the brain is implementing this.


He mentions how a lot of the functionality of the brain could be recovered, that you wouldn't get an avatar living in a virtual world, say, and that you wouldn't get an "emulation" -- you'd get an abstract model, or "imitation" of the brain. As I have said, that will get you pretty far. How far? See this updated version of my "Zombie AGI" post:

https://www.reddit.c...ual_assistants/

I conjecture you could use the model to build a really good "critic" to build an intelligent chatbot, for example.

....

Next up is this exciting work:

https://www.biorxiv....18/12/28/506956

Basically, they took a mouse and showed it images while recording its brain. They then took this stimuli & neural-response data, and used it to train a neural net with Deep Learning to predict the neural-response given an image as stimuli. Next, they sought to probe the neural net's generalization power: pick a neuron or small set of neurons; find or generate an image (that the model wasn't trained on), such that the model says the image should "maximally excite" these particular neurons (and has less effect on other neurons); and then show it to the mouse. Check to see if the model and mouse brain are in agreement regarding those neurons. The paper says that there is strong agreement.

They say:
 

Our work shows that these high-performing CNN black-box models of the visual system
do generalize and can make in silico inferences about non-trivial computational properties of V1 neurons.


It's one thing to produce a model that does pretty well on-average, and even works 98% of the time; it's quite another to produce a model that even nails down highly localized responses far out on the "long tail" of possible behaviors, as was done in this study.


It's worth mentioning that the model they used didn't just take into account V1 neurons:
 

The network also accounted for eye position and behavioral state (running, pupil dilation) of the animal (7), which we could measure but not control experimentally.



The same group had done that in earlier work.

....

Next, see this work:

https://www.research...tial_Recordings

A neural net given access to a small number of neuron state values of another neural net, can accurately learn to imitate that whole net, even with highly noisy recordings. What about brains?

....

Finally, see this recent work:

http://www.aclweb.or...hology/C18-1243

This is about applying brain data to NLP tasks. The data were from an experiment back in 2008. In several tasks they applied it to, they got very large improvements over modern methods. For example, using word2vec word vectors to compare most similar concrete nouns for "semantic similarity", they got a correlation of 0.83 with human judgment, while word2vec only gave 0.14. They got similar gains on the task of comparing least similar concrete nouns.

On other tasks, word2vec was better -- but not off-the-charts better. Keep in mind that the amount of data from that 2008 study was minuscule. The amount of data from the NeuroMod project will probably be several hundred times greater than from this 2008 study!
  • Yuli Ban likes this

#56
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
This is a very interesting piece of work:

https://www.scienced...053811918320937

The advent of brain reading techniques has enabled new approaches to the study of concept representation, based on the analysis of multivoxel activation patterns evoked by the contemplation of individual concepts such as animal concepts. The present fMRI study characterized the representation of 30 animal concepts. Dimensionality reduction of the multivoxel activation patterns underlying the individual animal concepts indicated that the semantic building blocks of the brain's representations of the animals corresponded to intrinsic animal properties (e.g. fierceness, intelligence, size).


One could imagine taking a long list of nouns, presenting them to people as their brain is scanned, performing "dimensionality reduction" on the brain responses to produce "brain word vectors", and then building a large "knowledge graph" for concepts. This knowledge graph would contain information about animals, plants, famous people, places, buildings, bridges, etc. Then, using some example (noun, attribute query, attribute) triples (e.g. query = "Are lions fierce?"; noun = lion, attribute = true/fierce) to train a classifier, you can predict a large number of attributes of a large number of concepts / nouns in the database.

With enough scanners, you could extract, say, 200-dimensional vectors for over 1 million concepts / nouns; and using readily available example triples from a few thousand nouns (far smaller than the 1 million), and maybe 200 attributes, you could determine the parameters in that classifier. This would give you, maybe, 200 million (noun, attribute query, attribute) triples in total, applying the classifier to the brain vectors.

Some attributes may not apply to certain classes of nouns, so you might want to use sets of attributes, instead of all 200 for each noun.

For concepts not among the 1 million, you could attempt to predict the brain response by expressing the concept as a combination of other concepts. For example, using word cooccurrence statistics, you could try to predict how new nouns relate to other ones; and then take some kind of weighted average of the ones in the dataset. This is similar to what was done in the 2008 work of a group at CMU (that included Tom Mitchell).

In this way, you could build a database of brain vectors for hundreds of millions of concepts -- maybe even billions.

Next, you could try to figure out how context influences things. e.g. whether we are talking about a biological concept in an informal or formal setting. This, too, can be done.

Bit by bit, one could build up a vast knowledge-base of concepts, that takes into account context, and returns human-like responses to queries. (Humans might get things a little bit wrong; but the machine should probably make the same mistakes, in order to understand human language better.)

....

But I guess this is fast becoming the "old way" of doing "language understanding". The trend seems to be away from breaking queries down into pieces and looking up the answers in a large knowledge graph, and towards just feeding in whole sentences to a neural net and having it sort things out. We're not there yet; but I think that will be the future. This, too, can be improved using brain data. One very low-level approach would be to replace the "word vectors" used in some of these approaches with "brain word vectors" (some of the neural net approaches work by first converting strings of words into word2vec or Glove word vectors, before feeding the stream into a neural net).
  • Yuli Ban and Alislaws like this

#57
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
One more piece of research to add:

Deep reinforcement learning from error-related potentials via an EEG-based brain-computer interface
 
https://ieeexplore.i...ocument/8621183

Basically, they had people watch virtual agents (robots) while recording subjects' brain activity using an EEG cap. If the agent didn't perform the way the subject expected, there was a detectable brain response that was used to make the agent work better next time.

The classification (of brain response) wasn't great; but with enough people contributing, rapid gains are possible.

Consider the following: let's say you have an addictive VR game that people play using a VR headset equipped with ear-EEG. Thousands of players watch virtual robots, and if they see the robot respond in a strange way, their brain reacts, and there is a detectable signal. In a very short time, tens of millions of signals are sent to the training algorithm to correct the robots. They rapidly get better, and soon behave in a graceful, human-like way.

With ear-EEG, the error from each brain will be large. But upon combining many thousands of brains together, the noise doesn't matter so much.

If people don't focus on the agents, but still see them, there will still be an error signal -- it will just have more noise than when they focus; but with thousands of people involved, that shouldn't be much of a problem.

The best part of all this is that it doesn't require actively trying to train the robot. There are error signals that are subconscious, that happen without your even trying.
  • Yuli Ban likes this

#58
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
The success of OpenAI's text-synthesis system perhaps makes you wonder if brain data (+stimuli) is still a good idea, as far as building advanced AI.  I think the answer is a resounding "YES", and will explain why.  I am on record, though, from months ago (and even years ago)  claiming that text data, alone, will take you very far.  So, I'm not opposed to it, and like all the progress.  
 
I also don't think text + video + audio is better than brain data.  The only advantage these sources have is that they already exist at large scale, while the BCI revolution hasn't even begun yet.  
 
The success of these large-scale projects should make one even more hopeful that brain imitation will be possible, given enough data.  If so much can be accomplished with text alone, doesn't it seem reasonable to believe data more directly related to the problem at hand (mind-building) would work even better?
 
Ok, so the reasons why brain data are better are:
 
* Brain data will almost certainly be a more compact representation of the knowledge you want the system to learn, on an example-by-example basis, but maybe not in terms of the total number of bytes.  There is a lot of approximate commonsense knowledge that can be extracted from billions of words using co-occurrence statistics; but you should only need a tiny fraction as many brain responses to words to get an equivalent amount of world knowledge from BCI data.  I would imagine brain data is at least 1000x more efficient as unstructured text, in terms of the number of responses you need to build your AI.
 
* Not only is it more efficient in terms of acquiring knowledge, but also in terms of acquiring skills.  BCI data contains detailed information about the step-by-step process your own brain uses to complete tasks.  Given only text data, the ML algorithms would have infer those steps; and so need a lot of examples to properly learn them.
 
* This example-efficiency means you don't need to rely on a large company like Google to supply that data; a not-too-large group could build their own datasets from scratch, using their own brains.  Large companies can't shut down progress by denying developers access to the data.  This can be both good bad from an AI safety perspective.  
 
* There is more -- much more -- to AI than building a system that can pass a conversational Turing Test.  There is also visual understanding, dexterous grasping, robot planning, and many other things.  These are things that can also be extracted from brain data.
 
* Finally, brain data is rather more natural to learn from than text statistics.  Not as many priors will be needed; and many common ML algorithms that imitate the brain will probably generalize much better than systems based on text, with its myriad exceptions to the rule and subtexts (hidden context and implications).  Imitating the brain is like trying to model the weather, fluid dynamics, and other physical systems.  
 
An example of the problem of generalization:  consider this conversation:
 
User: So what did you think of the game last night?
 
Computer:  I don't watch football.
 
User:  But it's not football season!
 
The computer could try to learn about sports seasons, and avoid this mistake; but how would it do this?  It could take a timid approach, and learn each individual month associated with each sport -- which would be possible, if you have enough text -- or, it could try to "fill in the gaps", by observing some months where football is played, and then guess that months in-between, that it hasn't seen examples of, probably play the same sports.  
 
This may work with football, but there may be other areas of knowledge where such a plan doesn't produce good guesses.  With brain imitation, though, since your priors are going to be a lot simpler and fewer in number, your guesses will almost always be right or nearly right, resulting in much better generalization performance.  
 
In fact, there are lots of experiments showing this to be the case:  Deep Learning models accurately predict neural responses far out on the "long-tail" of possible behaviors:
 

  • Yuli Ban likes this

#59
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
The COSYNE 2019 conference booklet has now been published online:

http://cosyne.org/co...rogram_book.pdf

Here is an abstract that resonates with what I discussed earlier about modelling whole brains of animals using Deep Learning, and includes David Sussillo as a coauthor:

T-43. Inferring brain-wide neuronal dynamics on single trials via neural stitching of many Neuropixels recordings

An emerging goal of systems neuroscience is to understand how complex neuronal dynamics enable the brain’s ability to select, plan, and execute adaptive behaviors. While recent progress has been made in understanding neuronal population dynamics in one or a few recorded brain regions, naturalistic behaviors require the coordination of many interconnected regions throughout the brain. Fortunately, it is now becoming possible to record densely throughout the brain with new electrophysiology technology, although not yet fully simultaneously. We begin to address the challenge of modeling global brain dynamics by developing an extension to LFADS (latent factor analysis via dynamical systems). As in LFADS, our method infers the latent, single trial dynamics of neural spiking data and performs neural dynamical stitching across separate recording sessions. Our extension leverages the brain-region location of each recorded neuron to construct region-specific latent dynamics as a subset of the global brain latent dynamics in the model.

We apply this method to a dataset comprising 23,881 neurons identified within 34 brain regions across 87 Neuropixels recording sessions in 21 mice performing an olfactory-decision task. We find that our modified, dynamically-stitched LFADS model can indeed model this large dataset at the single trial level. We find that (1) the model successfully recapitulates gating of the odor-response dynamics by a specific, persistent, and brainwide pattern of population activity that encodes motivational state at the single trial level, and (2) the satiety state of the animal is reliably encoded in the latent initial condition, and the odor identity in the model’s inferred inputs.

In summary, this modeling tool enables researchers to leverage non-overlapping electrophysiology datasets from across the brain to extract better estimates of the global latent dynamics that drive behavior. This approach further enables comparison of the contributions of individual brain regions to the global computation.


That's really impressive-sounding!

I'd like to see if it's possible to build an artificial animal model, with a virtual mouse that can walk around, sniff, use its whiskers, and so on. It wouldn't necessarily have to model all the mouse's higher cognitive functions to be a "success".

I suspect it is possible, and would be the most impressive feats of brain modelling yet tried. It would also signal that a similar feat may be possible with primates and even humans -- zombie apes and humans that can walk and respond to stimuli.
  • Yuli Ban likes this

#60
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,287 posts
This is a great article about Alex Huth's work:

http://neuroscience....n-in-the-brain/
 

Alex Huth, Assistant Professor of Computer Science and Neuroscience at the University of Texas at Austin, has an ambitious goal — to scan the brains of individual people for hundreds of hours to get fMRI data sets large enough to produce an accurate and detailed model of how language is represented in the brain.

....

AH: One of the things we did in the Gallant lab at Berkeley was really kind of different from the rest of the field. Instead of the standard MRI experiment [where] you take a bunch of people, scan each of them for maybe an hour, show them the same small set of stimuli, and average across these peoples’ brains to get some result.

What we did in the Gallant lab is take a smaller group of people, like 5-10 people, and scan each of them for many, many, many hours. In the paper that I published, it spanned from maybe 8-10 hours at least in the scanner, from which we got 2-3 hours of usable data. That’s a lot of time. Then we’d be able to do all these fantastic analyses. You could build these high-dimensional models because we’d have all this data from each person’s brain.

So one of the things I’m trying to do in my new lab is to take that idea and push it to the extreme. So ask — how much data can we get on a single subject? My goal is to have 200 hours from one person. So scan the same person over and over again, probably something like once a week for a couple years. The cool thing is that this allows us to really change how we think about the models that we would fit to this data.

Using our old approach of getting 2-3 hours per subject, we were kind of stuck in this mode of guess and check. We’d always have to guess — maybe this kind of thing is represented in the brain, maybe this kind of feature is important. Then we’d build models using that feature, then test to see if they work well.

But if you can get enough hours of data, enough data points, then we can let the data tell us what kind of features are important, instead of being forced to guess. So we can sort of flip the equation around. I think that’s really exciting, that’s the pivot point that we’re trying to get toward.

It’s getting a really, really big data set, getting enough that we can learn directly from that data what the features are, and that will tell us something about how the brain processes language. That’s the main thrust of my lab.



And next-gen BCIs will push that much further, since they will have much higher temporal resolution than fMRI, and spatial resolution in the ballpark of fMRI. It may make take a generation or two for BCIs to get to that level, but I could see it happening in a small number of years, given the rate of progress. When we can build good models of the brain's capacity to understand language, then we will be much closer to having machines with True Natural Intelligence.

It will be so much easier to build super-latge datasets that way.
  • Yuli Ban likes this





Also tagged with one or more of these keywords: AI, BCIs, Brain-Computer Interface, Artificial Intelligence

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users