[R] Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers

Submitted by currentscurrents t3_10ly7rw in MachineLearning

Dec 2022 paper from Microsoft research: https://arxiv.org/abs/2212.10559v2

>Large pretrained language models have shown surprising In-Context Learning (ICL) ability. With a few demonstration input-label pairs, they can predict the label for an unseen input without additional parameter updates. Despite the great success in performance, the working mechanism of ICL still remains an open problem. In order to better understand how ICL works, this paper explains language models as meta-optimizers and understands ICL as a kind of implicit finetuning.

235

Comments

You must log in or register to comment.

currentscurrents OP t1_j608oz5 wrote

TL;DR:

  • In-context learning (ICL) is the ability of language models to "learn from example" to perform new tasks just based on prompting. These researchers are studying the mechanism behind ICL.

  • They show that the attention layers allow transformers to implement a gradient descent optimization process at inference time. This mechanism produces very similar results to explicit optimization through fine-tuning, but was itself learned by optimization through gradient descent.

  • Based on this finding they apply momentum, a technique known to improve optimizers, to transformer attention layers. This produces a small-but-consistent improvement in performance on all tested tasks. They suggest that there are more improvements to be made by explicitly biasing transformers towards meta-optimization.

This reminds me of some meta-learning architectures that try to intentionally include gradient descent as part of inference (https://arxiv.org/abs/1909.04630) - the difference here is that LLMs somehow learned this technique during training. The implication is pretty impressive: at enough scale, meta-learning just emerges by itself because it's a good solution to the problem.

Other researchers are looking into ICL as well, here's another recent paper on the topic: https://arxiv.org/abs/2211.15661

94

lucidraisin t1_j61h7lf wrote

and one more paper along same lines! https://arxiv.org/abs/2212.07677

32

currentscurrents OP t1_j61ndkl wrote

Thanks for the link!

I think it's interesting that they spent so much time in the 90s trying to make meta-learning work, and now it appears emergently just from throwing scale at the problem.

33

DigThatData t1_j61zv3l wrote

Compute Is All You Need

38

endless_sea_of_stars t1_j627a9m wrote

Just rent out an AWS region for a month and you'll be good to go. Hold a couple bake sales to defray the cost.

13

robdogcronin t1_j61zvce wrote

That's the bitter lesson

18

currentscurrents OP t1_j623hb4 wrote

Yeah, but I want AI now. Not in 40 years when computers are 1000x better.

Also I'm not sure computers will be 1000x better in 40 years, Moore's law isn't what it used to be.

16

EarthquakeBass t1_j64jhk3 wrote

https://en.m.wikipedia.org/wiki/Huang%27s_law

A bit of marketing flair for sure, but I think at the crossroads of hardware improvements, ensembling, clever optimizations etc. we will keep improving models at a pretty darn fast pace. GPT-3 alone dramatically has improved the productivity of engineers, I’m sure of it.

3

throwaway2676 t1_j68vbfq wrote

> Not in 40 years when computers are 1000x better.

It won't take anywhere near that long. We've barely scratched the surface of ASICs and analog matrix multiplication, which is where the real fun is going to begin.

2

ElectronicCress3132 t1_j629tix wrote

> implement a gradient descent optimization process at inference time

Could you expand on what this means? At inference time, I thought all weights were frozen, so how could the attention layers be somehow performing gradient descent?

Edit: I read the paper in detail and understood it (walk through the math in Section 3). Basically, the sentence itself X has some weights that go through the attention layer (recall how attention works: it embeds the sentence, then multiplies it by key, value, query matrices). If you give it some examples, X', to learn from, well, of course there are going to be weights for both X, and X'. Turns out those weights for X' end up being equivalent to stepping in gradient descent.

24

throwaway2676 t1_j6d99fw wrote

So shouldn't this mean we can train transformers using forward passes alone? It seems that it wouldn't be too difficult to derive an algorithm that updates the attention weights based on these results, but I don't believe the authors mention the possibility.

1

master3243 t1_j61wtpt wrote

This is great work in collaboration with Microsoft Research. I'll have to read more than just the abstract and quickly skimming over it.

My only slight annoyance is the word "Secretly" in the title, I just feel a better word would be "implicitly" that would also be less "clickbait'-y

22

currentscurrents OP t1_j627rd0 wrote

Meh, transformers have been around for like 5 years and nobody figured this out until now.

I think this mostly speaks to how hard it is to figure out what neural networks are doing. Complexity is irrelevant to the training process (or any other optimization process), so the algorithms they implement are arbitrarily complex.

(or in practice, as arbitrarily complex as the model size and dataset size allow)

23

master3243 t1_j62aoln wrote

You're right they've been around for 5 years (and the idea for attention even before that) but almost every major conference still has new papers coming out giving more insight into transformers (and sometimes algorithms/methods older than it)

I just don't want to see titles flooded with terms like "secretly" or "hidden" or "mysterious", I feel it replaces scientific terms with less scientific but more eye-catchy ones.

Again I totally understand why they would choose this phrasing, and I probably would too, but in a blog post title not a research paper title.

But once again, the actual work seems great and that's all that matters really.

12

rjromero t1_j61ytag wrote

This is incredible research. Finally a lead on how we might get to "true" one shot / few shot learning.

20

currentscurrents OP t1_j62auto wrote

Yes, but I don't want to create too much optimism; meta-learning was also a promising lead when Schmidhuber wrote his PhD thesis.

Honestly, I'm not sure much has changed since then other than we got more compute power. Transformers are reportedly equivalent to 1990s meta-learning networks except that they run better on GPUs, and GPUs have gotten powerful enough to run them at very large scale.

25

cthorrez t1_j63uc5a wrote

I have an issue with the experiments.

> For ICL, we fix the number of demonstration examples to 32 and tune the random seed for each task to find a set of demonstration examples that achieves the best validation performance. For finetuning, we use the same demonstration examples for ICL as the training examples and use SGD as the optimizer

They go through a set of random seeds to pick the "best" possible samples for in context learning, and then use the same set of examples for fine tuning. I think this biases the results in favor of in context learning.

A more fair way to do this would be to use a truly random set of examples, or to use use the same approach and tune the seed to find the "best" set of examples for finetuning as well.

5

currentscurrents OP t1_j658kmf wrote

Interesting. That probably explains why ICL outperformed finetuning by so much in their experiments.

1

Complex_Candidate_28 t1_j675z5i wrote

The purpose of the experiments is not to compare the performance between them. The goal is to compare the mechanisms behind them. So it doesn't affect the conclusion itself. The point is to use the same set of examples for analysis.

1

cthorrez t1_j67aa39 wrote

If the goal is the mechanism rather than the performance why tune the seed for performance in the first place? The examples used doesn't change the mechanism.

2

Complex_Candidate_28 t1_j67aytx wrote

Because for small-size LMs, ICL is unstable, i.e., it sometimes degrades to classifying all examples into one category. The protocol tries to ensure analyzing ICL when it works well. (For much larger-size LMs, the performance variance would be much smaller, where this step can be ignored.)

1

cthorrez t1_j67csjx wrote

That's an interesting topic that I think deserves further investigation. On the surface it sounds like the size of the LM impacts the mechanism by which the LM is able to "secretly perform gradient descent".

Is finetuning similarly unstable for small sized LMs?

1

VisceralExperience t1_j61znkf wrote

The amount of blatant anthropomorphism that comes from AI researchers is so disgusting. Laymen knowledge about the state of the field is already twisted enough from reality, and the researchers are 100% to blame. Seriously, I'd like to see papers getting rejected for this delusional framing of results.

−19

currentscurrents OP t1_j620shg wrote

What? "Meta-optimization" is not a very anthropomorphic term, and certainly not something laymen would understand. Their approach is technical in nature and describes the limitations of current models in explicit detail.

20