Viewing a single comment thread. View all comments

seventyducks t1_j1z3w3a wrote

Reply to comment by comefromspace in [D] DeepMind has at least half a dozen prototypes for abstract/symbolic reasoning. What are their approaches? by valdanylchuk

I think there's a fallacy here - it may be the case that language involves all manner of symbol manipulation in its extended manifestation, but there is still considerable evidence that LLMs are not wholly capable of what we mean when we talk about language. There are many capacities still missing in even the most powerful LLMs. It may be the case that more data and more scale and some clever tricks will resolve these issues—though I am skeptical—but from what I have seen, LLMs thus far demonstrate a very limited capacity for 'symbol manipulation.' Namely, they show capacities for generation of statistically plausible sequences of letters, but fail in obvious ways on other sophisticated forms of symbolic manipulation and reasoning.

I'd be curious to hear if you agree, or perhaps if you think that the current limitations in symbol manipulation will be overcome with more scale on same architectures? This was a core question in the AGI Debate hosted by Montreal AI last week, and it seems experts on the subject are quite divided.

18

All-DayErrDay t1_j204aob wrote

What are the central limitations were considering here? Let’s define them in concrete terms.

3

Cheap_Meeting t1_j21v6mi wrote

I think the main limitations of LLMs are:

  1. Hallucinations: They will make up facts.
  2. Alignment/Safety: They will sometimes give undesirable outputs.
  3. "Honesty": They cannot make reliable statements about their own knowledge and capabilities.
  4. Reliability: They can perform a lot of tasks, but often not reliably.
  5. Long-context (& lack of memory): They cannot (trivially) be used if the input size exceeds the context length.
  6. Generalization: They often require task-specific finetuning or prompting.
  7. Single modality: They cannot easily perform tasks on audio, image, video.
  8. Input/Output paradigm: It is unclear on how to use them for tasks which don't have a specific inputs and outputs (e.g. tasks which require taking many steps).
  9. Agency: LLMs don't act as agents which have their own goals.
  10. Cost: Both training and inference incur significant cost.
3

Flag_Red t1_j22aiul wrote

Only #1 here really relates to their symbolic reasoning capabilities. It does imply that symbolic reasoning is a secondary objective for the models, though.

4

seventyducks t1_j21gedt wrote

To be honest I'm not going to spend a long time thinking it through and being intellectually precise for a Reddit comment, I'd recommend you check out the AGI Debate I mentioned above for experts' opinions.

−2

comefromspace t1_j1z64ux wrote

Language is syntax, and LLMs excel at it. I think it is interesting to note that GPT improved with learning programming because programming languages follows exact syntactic rules, which are rules of symbol manipulation. But it seems those rules are also great when applied to ordinary language which is much more fuzzy and ambiguous. transformers do seem to be exceptional at capturing syntactic relationships without necessarily knowing what it is that they are talking about (so, abstractly). And math is all about manipulating abstract entities..

I think symbol manipulation is something that transformers will continue to excel at. After all it's not that difficult either - Mathematica does it. The model may not understand the consequences of their inventions, but it will definitely be able to come up with proofs , models, theorems, physical laws etc. If the next GPT will be multi-modal, it seems it might be able to reason about its sensory inputs as well

−2

seventyducks t1_j1zcf8m wrote

>Language is syntax

Language is much more than syntax; if language as pure syntax is your starting point then it's not really a conversation worth having IMO.

11

alsuhr t1_j1zftk4 wrote

Language is an action we take to achieve some short- or long-term intent by affecting others' actions. It just so happens that text data is (mostly) symbolic, so it appears like only a problem of symbol manipulation. The text that these models are trained on are observations of language production, where utterances are generated from intent (e.g., wanting to convince someone of some argument, wanting to sell something to someone) and context (e.g., what you know about your interlocutor). This doesn't even cover vocal / signed communication, which is much more continuous.

Intent and context are not purely symbolic. Sure, with infinite observations, that generative structure would be perfectly reconstructable. But we are nowhere near that, and humans are completely capable of modeling that generative process with very little data and continuous input (which we learn to discretize).

10

maxToTheJ t1_j1zq0fa wrote

> Intent and context are not purely symbolic.

Yup . Thats why reasoning comes in and what makes what Demis from DeepMind said make sense

2

madnessandmachines t1_j1zkd85 wrote

Just want to reiterate: if you think language is just syntax, I'd recommend listening to some linguistics lectures or reading a book or two on the subject (i.e.: books on language, not on syntax or grammar). John McWhorter has some very approachable and eye-opening Audible courses that might change your perspective.

5

comefromspace t1_j1zm1fv wrote

I am aware of some of the philosophy of language, but i prefer to look at the neuroscientific findings instead. Language is a human construct that doesn't really exist in nature - communication does, which in humans is exchange of mental states between brains. The structure of language follows from abstracting the physical world into compact communicable units, and syntax is a very important byproduct of this process. I am more interested to see how hierarchical structure of language arises in these computational models like LLMs, which are open to empirical investigation. Most folk linguistic theories are high conjecture that has only circumstancial evidence.

−4

madnessandmachines t1_j1zok1s wrote

Linguistics is a field of study and analysis, not philosophy. And I am specifically talking about exploring the anthropological and ethnographical study of language which is where you might lose many of your assumptions. The way different languages work, how they change over time, is relevant to anyone working in NLP.

I would argue the number one fallacy of modern LLM design is people disregarding all we have come to know about language in favor of just hoping something interesting will emerge when we throw billions of parameters at it.

7

madnessandmachines t1_j1zousp wrote

Also “the structure of language follows from abstracting the world into compact communicable units” is itself a “folk theory” of languages. Many supposedly neuroscientific theories of language are little more than conjecture based on assumptions.

3

comefromspace t1_j23hsyi wrote

It is a conjecture that can be tested however, starting with artificial networks. I don't think it's folk theory because it s not mainstream at all

1