[D] DeepMind has at least half a dozen prototypes for abstract/symbolic reasoning. What are their approaches?

Submitted by valdanylchuk t3_zx7cxn in MachineLearning

In TED Interview on the future of AI from three months ago, Demis Hassabis says he spends most of his time on the problem of abstract concepts, conceptual knowledge, and approaches to move deep learning systems into the realm of symbolic reasoning and mathematical discovery. He says at DeepMind they have at least half a dozen internal prototype projects working in that direction:

https://youtu.be/I5FrFq3W25U?t=2550

Earlier, around the 28min mark, he says that while current LLMs are very impressive, they are nowhere near reaching sentience or consciousness, among other things, because they are very data-inefficient in their learning.

Can we infer their half dozen approaches to abstract reasoning from the research published by DeepMind so far? Or is this likely to be some yet unreleased new research?

DeepMind list many (not sure if all) of their papers here:

https://www.deepmind.com/research

I was able to find some related papers there, but I am not qualified to judge their significance, and I probably missed some important ones because of the less obvious titles.

https://www.deepmind.com/publications/symbolic-behaviour-in-artificial-intelligence

https://www.deepmind.com/publications/discovering-symbolic-models-from-deep-learning-with-inductive-biases

https://www.deepmind.com/publications/neural-symbolic-vqa-disentangling-reasoning-from-vision-and-language-understanding

https://www.deepmind.com/publications/learning-symbolic-physics-with-graph-networks

https://www.deepmind.com/publications/how-to-transfer-algorithmic-reasoning-knowledge-to-learn-new-algorithms

https://www.deepmind.com/publications/a-simple-approach-for-state-action-abstractionusing-a-learned-mdp-homomorphism

Can anyone help summarize the approaches currently considered promising in this problem? Are we missing something bigger coming up behind all the hype around ChatGPT?

203

Comments

You must log in or register to comment.

EducationalCicada t1_j1yux4o wrote

Previously from Deep Mind in the domain of symbolic reasoning:

Making Sense Of Sensory Input

>This paper attempts to answer a central question in unsupervised learning:what does it mean to "make sense" of a sensory sequence? In our formalization,making sense involves constructing a symbolic causal theory that both explains the sensory sequence and also satisfies a set of unity conditions. The unity conditions insist that the constituents of the causal theory -- objects,properties, and laws -- must be integrated into a coherent whole. On our account, making sense of sensory input is a type of program synthesis, but it is unsupervised program synthesis.
>
>Our second contribution is a computer implementation, the Apperception Engine, that was designed to satisfy the above requirements. Our system is able to produce interpretable human-readable causal theories from very small amounts of data, because of the strong inductive bias provided by the unity conditions.A causal theory produced by our system is able to predict future sensor readings, as well as retrodict earlier readings, and impute (fill in the blanks of) missing sensory readings, in any combination.
>
>We tested the engine in a diverse variety of domains, including cellular automata, rhythms and simple nursery tunes, multi-modal binding problems,occlusion tasks, and sequence induction intelligence tests. In each domain, we test our engine's ability to predict future sensor values, retrodict earlier sensor values, and impute missing sensory data.
>
>The engine performs well in all these domains, significantly out-performing neural net baselines. We note in particular that in the sequence induction intelligence tests, our system achieved human-level performance. This is notable because our system is not a bespoke system designed specifically to solve intelligence tests, but a general-purpose system that was designed to make sense of any sensory sequence.

Edit: Also check out the followup paper:

Making Sense Of Raw Input

36

valdanylchuk OP t1_j1z073m wrote

Very cool! And this paper is from 2019-20, and some of those I listed in my post are from 2018-19. I wonder how many of these turned out dead ends, and how far did the rest go by now. Papers for major conferences are often preprinted in advance, but sometimes DeepMind also comes out with something like AlphaGo or AlphaFold on their own schedule. Maybe some highly advanced Gato 2.0 is just around the corner?

9

evanthebouncy t1_j1zo0rp wrote

hey, I work on program synthesis, which is a form of neuro-symbolic reasoning. here's my take.

the word "neuro-symbolic" is thrown around a lot, so we need to first clarify which kinds of work we're talking about. broadly speaking there are 2 kinds.

  1. neuro-symbolic systems where the symbolic system is _pre-established_ where the neuro network is tasked to construct symbols that can be interpreted in this preexisting system. program synthesis falls under this category. when you ask chatgpt/copilot to generate code, they'll generate python code, which is a) symbolic and b) can be interpreted readily in python
  2. neuro-symbolic systems where the neural network is tasked to _invent the system_. take for instance the ARC task ( https://github.com/fchollet/ARC ), when humans do these tasks (it appears to be the case that) we first invent a set of symbolic rules appropriate for the task at hand, then apply these rules

I'm betting Demmis is interested in (2), the ability to invent and reason about symbols is crucial to intelligence. while we cannot understate the value of (1) , reasoning in existing symbolic system is immediately valuable (e.g. copilot).

some self-plug on my recent paper studying how people invent and communicate symbolic rules using natural language https://arxiv.org/abs/2106.07824

26

yazriel0 t1_j237x73 wrote

> we cannot understate the value of (1) , reasoning in existing symbolic system

ofc. and (1) may be a good way to bootstrap (2) ..

why arent we seeing more (un)supervised learning on code? perhaps with handcrafted auxiliary tasks.

when will this loop exit? how much memory will this function allocate? etc, etc. this seems to be a huge underutilized dataset.

am i missing something? (yes, its a lot of compute)

1

comefromspace t1_j1ytlcw wrote

I don't know but it seems like LLMs will get there faster as soon as they become multimodal. Language is already symbol manipulation.

11

seventyducks t1_j1z3w3a wrote

I think there's a fallacy here - it may be the case that language involves all manner of symbol manipulation in its extended manifestation, but there is still considerable evidence that LLMs are not wholly capable of what we mean when we talk about language. There are many capacities still missing in even the most powerful LLMs. It may be the case that more data and more scale and some clever tricks will resolve these issues—though I am skeptical—but from what I have seen, LLMs thus far demonstrate a very limited capacity for 'symbol manipulation.' Namely, they show capacities for generation of statistically plausible sequences of letters, but fail in obvious ways on other sophisticated forms of symbolic manipulation and reasoning.

I'd be curious to hear if you agree, or perhaps if you think that the current limitations in symbol manipulation will be overcome with more scale on same architectures? This was a core question in the AGI Debate hosted by Montreal AI last week, and it seems experts on the subject are quite divided.

18

All-DayErrDay t1_j204aob wrote

What are the central limitations were considering here? Let’s define them in concrete terms.

3

Cheap_Meeting t1_j21v6mi wrote

I think the main limitations of LLMs are:

  1. Hallucinations: They will make up facts.
  2. Alignment/Safety: They will sometimes give undesirable outputs.
  3. "Honesty": They cannot make reliable statements about their own knowledge and capabilities.
  4. Reliability: They can perform a lot of tasks, but often not reliably.
  5. Long-context (& lack of memory): They cannot (trivially) be used if the input size exceeds the context length.
  6. Generalization: They often require task-specific finetuning or prompting.
  7. Single modality: They cannot easily perform tasks on audio, image, video.
  8. Input/Output paradigm: It is unclear on how to use them for tasks which don't have a specific inputs and outputs (e.g. tasks which require taking many steps).
  9. Agency: LLMs don't act as agents which have their own goals.
  10. Cost: Both training and inference incur significant cost.
3

Flag_Red t1_j22aiul wrote

Only #1 here really relates to their symbolic reasoning capabilities. It does imply that symbolic reasoning is a secondary objective for the models, though.

4

seventyducks t1_j21gedt wrote

To be honest I'm not going to spend a long time thinking it through and being intellectually precise for a Reddit comment, I'd recommend you check out the AGI Debate I mentioned above for experts' opinions.

−2

comefromspace t1_j1z64ux wrote

Language is syntax, and LLMs excel at it. I think it is interesting to note that GPT improved with learning programming because programming languages follows exact syntactic rules, which are rules of symbol manipulation. But it seems those rules are also great when applied to ordinary language which is much more fuzzy and ambiguous. transformers do seem to be exceptional at capturing syntactic relationships without necessarily knowing what it is that they are talking about (so, abstractly). And math is all about manipulating abstract entities..

I think symbol manipulation is something that transformers will continue to excel at. After all it's not that difficult either - Mathematica does it. The model may not understand the consequences of their inventions, but it will definitely be able to come up with proofs , models, theorems, physical laws etc. If the next GPT will be multi-modal, it seems it might be able to reason about its sensory inputs as well

−2

seventyducks t1_j1zcf8m wrote

>Language is syntax

Language is much more than syntax; if language as pure syntax is your starting point then it's not really a conversation worth having IMO.

11

alsuhr t1_j1zftk4 wrote

Language is an action we take to achieve some short- or long-term intent by affecting others' actions. It just so happens that text data is (mostly) symbolic, so it appears like only a problem of symbol manipulation. The text that these models are trained on are observations of language production, where utterances are generated from intent (e.g., wanting to convince someone of some argument, wanting to sell something to someone) and context (e.g., what you know about your interlocutor). This doesn't even cover vocal / signed communication, which is much more continuous.

Intent and context are not purely symbolic. Sure, with infinite observations, that generative structure would be perfectly reconstructable. But we are nowhere near that, and humans are completely capable of modeling that generative process with very little data and continuous input (which we learn to discretize).

10

maxToTheJ t1_j1zq0fa wrote

> Intent and context are not purely symbolic.

Yup . Thats why reasoning comes in and what makes what Demis from DeepMind said make sense

2

madnessandmachines t1_j1zkd85 wrote

Just want to reiterate: if you think language is just syntax, I'd recommend listening to some linguistics lectures or reading a book or two on the subject (i.e.: books on language, not on syntax or grammar). John McWhorter has some very approachable and eye-opening Audible courses that might change your perspective.

5

comefromspace t1_j1zm1fv wrote

I am aware of some of the philosophy of language, but i prefer to look at the neuroscientific findings instead. Language is a human construct that doesn't really exist in nature - communication does, which in humans is exchange of mental states between brains. The structure of language follows from abstracting the physical world into compact communicable units, and syntax is a very important byproduct of this process. I am more interested to see how hierarchical structure of language arises in these computational models like LLMs, which are open to empirical investigation. Most folk linguistic theories are high conjecture that has only circumstancial evidence.

−4

madnessandmachines t1_j1zok1s wrote

Linguistics is a field of study and analysis, not philosophy. And I am specifically talking about exploring the anthropological and ethnographical study of language which is where you might lose many of your assumptions. The way different languages work, how they change over time, is relevant to anyone working in NLP.

I would argue the number one fallacy of modern LLM design is people disregarding all we have come to know about language in favor of just hoping something interesting will emerge when we throw billions of parameters at it.

7

madnessandmachines t1_j1zousp wrote

Also “the structure of language follows from abstracting the world into compact communicable units” is itself a “folk theory” of languages. Many supposedly neuroscientific theories of language are little more than conjecture based on assumptions.

3

comefromspace t1_j23hsyi wrote

It is a conjecture that can be tested however, starting with artificial networks. I don't think it's folk theory because it s not mainstream at all

1

lorepieri t1_j1z4zp5 wrote

Some thoughts and then some references to neuro-symbolic approaches:

The reality is that AGI has become (always been?) an engineering problem [For intellectual honesty: this is not consensus, e.g. among neuroscientists. See comments below]. Many times in the past we have seen less theoretically scalable methods outperform more principles ones, so nobody can predict which one will win in the short term. LLMs are promising since they can leverage all the hardware acceleration and the pre-existing work of different fields (NLP, Computer Vision, RL). So it may very well be that DL will be enough to achieve great results and more investment and optimisation will pile-in, making symbolic approaches comparatively less attractive to fund in the short term.

Who knows, maybe the right symbolic architecture has already been proposed 20-30 years ago and nobody took the effort to put into a modern GPU accelerated codebase.

https://arxiv.org/abs/2012.05876 Neurosymbolic AI: The 3rd Wave

https://arxiv.org/abs/2105.05330 Neuro-Symbolic Artificial Intelligence: Current Trends

https://arxiv.org/abs/2002.00388 A Survey on Knowledge Graphs: Representation, Acquisition and Applications

11

valdanylchuk OP t1_j1zb36a wrote

> Who knows, maybe the right symbolic architecture has already been proposed 20-30 years ago and nobody took the effort to put into a modern GPU accelerated codebase.

I also half-expect that in ten years, what current LLMs do on racks of GPUs, will fit in a phone chip, because many advances in efficiency come from utilizing old simple techniques like Monte Carlo and nearest neighbors.

10

lorepieri t1_j1zniuo wrote

Exactly, it's not just a matter of software architecture, but also of preexisting optimised libraries, hardware acceleration, economic incentives, funding. Very hard to predict how it will end up.

5

master3243 t1_j20v8i9 wrote

> The reality is that AGI has become (always been?) an engineering problem.

I would not state this as fact, I'd say the majority of neuroscientists and cognitive scientists will disagree with this (or say we don't know yet), and a fair number of AI researchers would too.

I doubt any but a few researchers would be comfortable saying "Yes it's definitely an engineering problem".

5

lorepieri t1_j23iett wrote

You are correct, edited to clarify that this is not consensus.

1

ReasonablyBadass t1_j20vrrq wrote

I find the data efficiency argument weird. We consider a human fully formed at around twenty years of age. Is that really all that "data efficient"?

7

elsjpq t1_j21s4eb wrote

Well you might wanna include the couple billion years of evolutionary selection as training time as well. Otherwise, there's a ton of stuff already "baked in" to the model.

10

Cheap_Meeting t1_j21q96n wrote

Yes, they are trained on a much larger amount of language data than a human sees in their lifetime.

However, I would argue that it's a worthwhile trade-off. Computers can more easily ingest a large amount of data. Humans get feedback from the environment (like their parents), can cross-reference different modalities, and have inductive biases.

2

wise0807 t1_j1zsc50 wrote

Wow, thanks so much for posting this. I myself am focussed on new mathematical models for AI and this is exactly what I needed.

3

TheSuperSam t1_j201bzh wrote

RemindMe! 5 days

2

Just_CurioussSss t1_j1zeg78 wrote

We never know. Imo LLM will grow exponentially. Today, they're data inefficient, but someday (definitely not tomorrow lol), more sentient than we can ever imagine.

−1