Jump to content

Welcome to FutureTimeline.forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. If you already have an account, login here - otherwise create an account for free today!
Photo

The future of machines that can "reason"


  • Please log in to reply
8 replies to this topic

#1
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts
Have a listen to this interview with OpenAI's Greg Brockman, that was posted just today:
 
YouTube video
 
Near the end, beginning 1 hour, 17 minutes, 29 seconds in:
 

Brockman:  Ilya and I are kicking off a new team, called the "reasoning team", and this is to really try to tackle, "How do you get neural networks to reason?"  And we think this will be a long-term project.  It's one we are very excited about.
 
Lex:  In terms of reasoning -- a super-exciting topic -- what kind of benchmarks, what kind of tests of reasoning would you envision?
 
Brockman:  Theorem-proving.

 
He also mentions how "programming", "security analysis". and theorem-proving all capture the same core processes of reasoning.  (In fact, if you can get a machine to prove theorems, you can transform the solution into a method to write code.  I've explained this before.)
 
Coincidentally, Deepmind released a dataset just today on getting a machine to solve basic math problems:
 
https://twitter.com/...420454860083201
 

Today we're releasing a large-scale extendable dataset of mathematical questions, for training (and evaluating the abilities of) neural models that can reason algebraically.


Reasoning algebraically (i.e. solving math exam problems) is very, very far from actual theorem-proving, which is orders of magnitude harder; but it's a step in that direction.

It seems like there is something in the air -- teams are now attempting to tackle this very hard problem with Deep Learning... a problem that skeptics have said it shouldn't be able to solve!

As I've said before, mathematical theorem-proving is an excellent benchmark task to training machines to do complex reasoning. It's great, because it's domain-limited, and doesn't contain all the messiness we see in the real world. AND, if they can get a machine to solve IMO-style problems, say, then probably they can tweak their program to solve problems in pretty much any axiomatic system -- including ones important to biology, physics, medicine, you name it. This won't quite get us to an "automated scientist", though, since a large part of science is dealing with the messy, real world, where you have to come up with a reasonable axiom system.

(You could argue, perhaps, that coming up with good axioms is, itself, a reasoning process that has its own meta-system of axioms (quantifying over a higher-order set of objects); but it might be really difficult to figure out what these are -- and it might involve a lot of them, whereas many mathematical systems have only very few axioms.)

Still, if mathematical theorem-proving becomes the new "Atari benchmark" for OpenAI and Deepmind, this one will be one that actually gives real-world benefits. It will make scientific progress go a lot faster.

....

Another thing worth pointing out from that interview with Brockman: he was asked whether just scaling-up GPT-2 would lead to a system that can pass a Turing Test, and was a little evasive. He said that he thinks that a true Turing Test would require the system to not just hold a conversation, but to also do reasoning of an indeterminate length (GPT-2 does bounded-depth logic / "reasoning"), and should also have the ability to learn as you converse with it. He also mentioned that it's not clear what the limits are of its world-modeling / commonsense reasoning, based on pure text training. Reasoning and learning (not at the training phase, but during the conversation phase) will be things that have to be added.

I suspect, though, that just given the bounded-depth logical inference and limited learning ability language models currently possess, with enough data, existing language models would go pretty far towards convincing the average human that they aren't talking to a machine.

He mentions how for 2019 OpenAI wants to scale up language modelling 100x to 1000x to see what will happen. By the sound of it, they want to build far larger models than even GPT-2-large -- say, GPT-2-HUGE.
  • Yuli Ban likes this

#2
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts
Here's the kind of thing you could use a theorem-prover to do: let's say you have a theory that electrodynamics and electrostatics can be modeled as a statistical process involving tile-counting -- like, say, in this paper:

https://www.research..._electrostatics

However, deriving the formulas for the tile counting is very difficult. It takes many pages of hard q-series manipulations.

If you had a program that could prove theorems, and it could do it quickly even for hard problems, it could derive these formulas for you in a matter of minutes or hours! And, then, if you didn't get the result you were aiming for, you could tweak the model until it does. In the end, you would have a new way of thinking about electrostatics and electrodynamics.

There are all kinds of problems in physics like that.

Another example: if you find the right "construction" in the theory of superconductivity, perhaps you would have a structure that is a good candidate for a room-temperature superconductor. And, then, you could try to cook it up in the lab. Coming up with that structure is an "existence proof" problem amenable to automated theorem-proving, if you can set up the axioms.

....

In medicine, if you could axiomatize systems in the human body, you could use a reasoner to work out a set of "interactions" or "interventions" that cause the entire system to behave in certain ways, resulting in the cure to a disease. Probably medicine would have a very large number of axioms and assumptions, and the kind of reasoning employed would be probabilistic, rather than deterministic. So, for example, you would only be able to say that a particular intervention cures the disease with probability 90%. But for many diseases that would be a game-changer.
  • Yuli Ban likes this

#3
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts
Here's a zdnet article about that Deepmind "algebraic reasoning" work:

https://www.zdnet.co...gh-school-math/

The basis for the questions was "a national school mathematics curriculum (up to age 16), restricted to textual questions (thus excluding geometry questions), which gave a comprehensive range of mathematics topics that worked together as part of a learning curriculum." They enhanced that basic curriculum, they write, with questions that "offer good tests for algebraic reasoning."

To train a model, they could have given some neural net math abilities, they note, but the whole point was to have it start from nothing and build up a math ability. Hence, they went with more or less standard neural networks.


Look at the chart -- that reminds me of the Atari games charts (to see how things are progressing on different types of games). The Transformer neural net, which is the type used for OpenAI's GPT-2, does pretty well across a range of problems. However, they say:

They conclude that while the Transformer neural net they build performs better than the LSTM variant, "neither of the networks are doing much "algorithmic reasoning," and "the models do not learn to do any algebraic/algorithmic manipulation of values, and are instead learning relatively shallow tricks to obtain good answers on many of the modules."


Still, if you feed the thing enough data, it may not matter, as far as getting good performance.

It will be interesting to see if teams can find some tweaks to make the thing generalize much better. If they can solve these high school (mostly grade school) problems to a high degree of proficiency, honestly (i.e. it generalizes), perhaps people will bump up the complexity and see if it can prove simple theorems... then, more complicated theorems... then, IMO-style theorems... and, finally, research-level math.
  • Yuli Ban likes this

#4
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts
Let's suppose OpenAI is partially successful, and in the next 3 years, say, they build a system that can solve IMO-style problems, and even some of the easier research-level math poblems -- say, where 5 pages or less of tricky, clever reasoning is required. Think what that would mean, in terms of employment:

I think it would signal that the days are numbered for a particular class of "geeks" who make their living from their analytical skills applied to specific classes of problems. For example:

* Computer security analysts.

* Certain classes of programmers -- e.g. ones whose jobs can be described as "highly technical work", whereby they take an existing, inefficient solution, and then make it work a lot quicker through various tricks. Or, programmers that take a well-defined problem, and then write an efficient, long program to solve it. Any kind of programming that relies on judgment and creativity -- e.g. where you have to choose a good user interface as part of the "solution" -- or where there is a lot of ambiguity and leeway in terms of how you approach it, will be safe.

* When combined with advances in Natural Language Processing (e.g. further developments in "pre-trained models" like BERT), it could also be the beginning of the end of the Market Research Analyst: you might have a set of economic and user behavioral "axioms" to work with, a large knowledge graph, and a set of mathematical and statistical modelling axioms; and then the job is to construct an analysis of certain companies, based on current data on how they are performing. The output of the analysis could be fed into a report-generator, like the ones Narrative Science has built.


I think the current consensus belief is that these jobs are safe for many, many years; but maybe they are all wrong, and one-trick-pony geeks (who take a well-defined, technical assignment as input, and then output a report, and never have to talk to another soul) could be sent to the soup kitchens sooner than we think. If OpenAI is successful about math theorem-proving in the next 3 years, I would say that within 5 to 10 years these jobs would be seriously threatened. OpenAI wouldn't be the source of the threat; but their work would signal that automating reasoning is easier than is widely believed -- and would bring all kinds of companies out of the woodwork to build their own next-gen reasoning engines.

I also think that if text-synthesis continues to improve (e.g. if OpenAI's scaling-up of GPT-2 by 100x to 1000x dramatically improves upon GPT-2-large), then competent writers of various sorts could feel the heat. It probably wouldn't have much effect on the William Gibsons of the world; just your middling pulp fiction writer. To look at it another way: it would mean that almost anybody could become a writer. They would just feed in a prompt, and out would pop a pretty good short story that could be cleaned-up a bit, and sent to a magazine. Maybe you'd have to try like 10 different versions, until you got one that was really good; but you wouldn't have to do the actual writing. You would only have to be a decent reader, and have good artistic judgment.

If Natural Language Generation and processing gets good enough, then when combined with "reasoning", another set of jobs could be automated. Some more classes of programmers could become unnecessary -- e.g. the program specifications could be a little fuzzier, and the system could translate that into a reasonable, exact formulation, just as language models can write stories when given a prompt. Might take an additional 5 years, beyond those 5 to 10 years -- But who knows? Could happen a lot quicker.

The kind of job that will endure will be one that requires a lot of different skills -- part writer, part secretary, part researcher, part counselor, part teacher, etc. Also, jobs that require hard-to-automate physical labor, like restaurant waiter (part of which is about making people feel welcome and treating them well), will endure.

But for a large class of "smart" professionals, the soup kitchen calls... "I'm sorry, but we don't need your services anymore. Clean out your desk and leave the premises."

Addendum: I should also point out that aesthetic and human judgement could be transferred to a machine using BCIs, or even just using very large amounts of human-generated media. So, even those programming jobs that require judgment could be automated away. Programming could, then, become a game of coding up axioms and specifications to describe all the libraries and input and output channels (e.g. from sensors) of the system. There probably would also still be a lot of very high-level stuff you need humans for -- e.g. in computer game design. But I would imagine a lot of business coding, for example, could be automated away, given the axioms and specifications.

#5
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts

New Google research on automated theorem-proving:

 

https://arxiv.org/abs/1904.03241

 

I can't really tell why people haven't yet trained neural net based systems to totally crack this important problem.  Probably it has more to do with producing very large datasets or number of training examples, than coming up with a good model.  There are far fewer proved theorems coded up to work with, than there is free text. 

 

If they can find a way to generate large synthetic datasets, then they might totally crack the problem.  I think this should be feasible.  


  • Yuli Ban likes this

#6
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts
I want to mention one more thing, regarding solving math problems: that DeepMind paper, where they train a giant network to solve simple math problems does pretty well (as I mentioned), but it was found to basically just memorize templates -- and isn't always so great at arithmetic. It was seen to be a partial success, and partial failure. But I think what this misses is that that's what most reasoning on math exams are like by humans!

I used to think that when people learn a subject, that to do it properly, they have to "learn how to think". In education we hear that constantly -- "We want them to think." But what students actually learn is a set of templates and terms at a superficial level; they learn "types of problems" and "solution templates", and have a shaky familiarity with technical terms, and then feed in the problem details to the template, and out pops the answer. The "we want them to think" is bullshit, if we're talking about exams -- proper thinking takes time, and you can't really test it in a 1 hour exam. Some small percent of students really do think, but most just learn templates. And I'm even talking about the highly intelligent students. (The truly gifted students are a different story.)

If I were to give an exam with problems that deviate even a little from homework problems, even many smart students would not be able to solve them.

What tends to happen is that, some time later, after they've learned the templates (and gotten their "A" on the exam), is that the templates start to melt together -- the students learn to "generalize" -- and then they really do acquire the capacity to solve novel problems, by combining together what they know in complex ways.

It's the same thing with learning language: many start by learning phrases; then, they learn to combine phrases; and finally, it all starts to melt together, and pretty soon they are fluently communicating in the new language. They don't need phrases anymore.

I used to blame people for not "thinking" (having been told this is what we want to see); but I don't anymore. I realize that it's just how the learning process works. It's not their fault.

....

While I'm on the subject: many high-level technical books are badly written, because they don't take into account how people actually learn. They are written like how language books used to be written, that were very heavy on grammar, and light on lots and lots of real-world examples. Technical books tend to be "bottom up", starting with very precise definitions, and then giving the absolute most general introductory lemmas with few examples (just enough to illustrate the basic "language game" involved; and most of the rest left as exercises). Very little discussion is given to what the stuff is actually useful for, and how to recognize when it's useful -- that kind of high-level meta-analysis / meta-thinking is discouraged; it leaves the pure Platonic realm being constructed before our eyes, and ventures too much into the "real world" (where people have "interests" and "careers").
  • Yuli Ban likes this

#7
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts
Another aspect of reasoning that isn't widely appreciated is that: people make mistakes. Lots of mistakes. You just wouldn't believe how many the greatest minds make on the way to a solution!

Someone who hasn't done serious research probably thinks that it involves: person A presents a crisp, perfectly-formed, correct statement for how to approach a problem. Person B then improvises on that, and presents a crisp, perfectly-formed, correct statement of how to take that further, getting a little closer to the answer. Then person C builds on that, with another crisp, perfectly-formed, correct statement, and it continues until the problem is solved.

The early Polymath projects (which are a community-wide effort to solve various math problems) dispelled that for many people (not me -- I am an old guy, remember, and was disabused of that myth long before those projects existed). There, they saw giants in the field make foolish errors, make many false starts, corrected themselves multiple times -- slowly, but surely, making their way to a solution, despite errors.

I remember when the Polymath projects first got going, some ph.d. students from U. C. Berkeley wrote something like, "Every grad student should see that! They should see how messy the process is!" They were under the illusion that research is a sequence of crisp, perfectly-formed, correct statements. That illusion exists, because they are used to seeing the final product -- a journal article, full of crisp, perfectly-formed, correct statements; or perhaps the way it appears in textbooks or in carefully-prepared lectures. Blog postings and comments perhaps also reinforce the myth, as they, too, are often relatively unmessy (depending on the blog). The difference with Polymath is that there is a time pressure -- people are attempting to solve the problem as quickly as possible, and that tends to bring out more what research looks like in real-time, in someone's office at a whiteboard.

This messiness is possibly a weakness of human cognition, that will make it easier to build machines to prove theorems. But maybe it actually is a strength -- maybe the messiness reflects a creative mind, that doesn't too quickly discount ideas. A more self-conscious mind might hold onto the idea, to see if it's "stupid" first, and not make as much progress -- instead of saying, "Well, it doesn't work, but feels like the right approach, so let's play with it a little."; they say, "Well, it doesn't work. The end."
  • Yuli Ban and Alislaws like this

#8
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts
This interview with Google's Christian Szegedy is really good:
 
https://youtu.be/p_UXra-_ORQ

He says basically the same thing I said above:
 

As I've said before, mathematical theorem-proving is an excellent benchmark task to training machines to do complex reasoning. It's great, because it's domain-limited, and doesn't contain all the messiness we see in the real world. AND, if they can get a machine to solve IMO-style problems, say, then probably they can tweak their program to solve problems in pretty much any axiomatic system -- including ones important to biology, physics, medicine, you name it. This won't quite get us to an "automated scientist", though, since a large part of science is dealing with the messy, real world, where you have to come up with a reasonable axiom system.


Szegedy's version of this begins 17:00 into the interview, where he says:
 

Once you have the first step, which is a super-human mathematician, you can infuse more domain knowledge, and then you can do software synthesis with it... [etc. etc.]



#9
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPip
  • 1,274 posts
This is a very interesting paper by some people at Google:

https://arxiv.org/abs/1909.11851v1

It's an attempt to see if they can give machines a human's intuitive way of thinking about mathematics, and also reduce the amount of hard symbolic tools or modules you have to build into a theorem-prover (in other words, make it as close to 100% Deep Learning as you can get away with).

Essentially, they wonder if it's possible for neural nets to do sequential symbolic reasoning in "latent space" in such a way that the representations don't degrade. A more down-to-earth (though not really accurate) take: is it possible to train a neural net to combine "thoughts" together in the way that humans do when thinking about math, so that those "thoughts" don't lose focus and succumb to entropy? -- you see, when humans think about math, we don't think in terms of symbolic logic; we think in pictures, ideas, and other things; and we seem to be good at keeping the thoughts together over many steps. Doing math in "latent space" is sort of like operating with / combining "thoughts" to produce new ones.

I see a big future in this sort of work. There are still several more steps before we see machines beating mathematicians at proving deep theorems; but those steps might could be breached fairly soon, if they can find a way to build large synthetic datasets to train models on, to give them a mathematician's intuitive understanding, and the ability to combine "thoughts" together.
  • Yuli Ban likes this




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users