This is an interesting paper:https://www.ncbi.nlm...les/PMC4708087/
Complex animal behaviors are likely built from simpler modules, but their systematic identification in mammals remains a significant challenge. Here we use depth imaging to show that three-dimensional (3D) mouse pose dynamics are structured at the sub-second timescale. Computational modeling of these fast dynamics effectively describes mouse behavior as a series of reused and stereotyped modules with defined transition probabilities. We demonstrate this combined 3D imaging and machine learning method can be used to unmask potential strategies employed by the brain to adapt to the environment, to capture both predicted and previously-hidden phenotypes caused by genetic or neural manipulations, and to systematically expose the global structure of behavior within an experiment. This work reveals that mouse body language is built from identifiable components and is organized in a predictable fashion; deciphering this language establishes an objective framework for characterizing the influence of environmental cues, genes and neural activity on behavior.
Another study, by some of the same group, wherein they used video of mouse behavior to train a Deep Learning + probabilistic graphical model to predict future mouse video:https://arxiv.org/abs/1603.06277
We propose a general modeling and inference framework that composes probabilistic graphical models with deep learning methods and combines their respective strengths. Our model family augments graphical structure in latent variables with neural network observation models. For inference, we extend variational autoencoders to use graphical model approximating distributions with recognition networks that output conjugate potentials. All components of these models are learned simultaneously with a single objective, giving a scalable algorithm that leverages stochastic variational inference, natural gradients, graphical model message passing, and the reparameterization trick. We illustrate this framework with several example models and an application to mouse behavioral phenotyping.
And there was also a study on predicting dog behaviors that people might have seen:https://arxiv.org/abs/1803.10827
Again, no brain data was used. If Deep Learning can be used to train a neural net to model zombie-like mouse video given only behavioral data, imagine what it could do if you just give it a little brain data!
There have also been a few studies that use brain data, as I mentioned earlier in this thread. For example, there was a study that used Deep Learning + brain data + behavioral data to jointly model mouse whisker movements, running on a track-pad, pupil dilation, visual fixations, and visual cortex processing. It didn't model smell, complex planning, location tracking, and many other things.
Another thing worth remarking on, that is tangential to this thread: not only can mouse motions be broken up into "motion syllables and phrases", but the same is true of human motion. It's also true of conversation at not only the basic level, but also at a high level. Some of my relatives, for example, have such modular, stereotyped conversations that I have been writing down pat answers to some of their questions -- and when they ask me again, I just show them the answer I wrote down. I also know people who have more free-form conversations; but I bet even they can be broken down into basic components -- though, different people will have different associated components.
This gives me hope that, even if it takes a while until machines can pass a Turing Test, high-performing conversational agents built using a "dialog state graph" with a gargantuan number of states, and with a high level of granularity, will make it possible for machines to have human-like conversations near-term, so long as one doesn't expect deep, philosophical conversations where people generate long monologues, and expect their listeners to follow every word.