Viewing a single comment thread. View all comments

-xylon t1_izx7jbc wrote

Reply to comment by IntelArtiGen in [D] G. Hinton proposes FF – an alternative to Backprop by mrx-ai

Training in that manner tends to "forget" previous knowledge in the net.

6

IntelArtiGen t1_izxdej3 wrote

I agree it's unperfect, as we are. When I tried to do it, I was still able to maintain a bit of knowledge in the network but I had to continously re-train on previous data.

It's hard to do "info1,2,3 => train => info4,5,6 => train => info7,8,9 => train [etc.]" and have the model remember info1,2,3

But you can do "info1,2,3 => train => info4,5,1 => train => info6,7,2 => train [etc.]". I used a memory to retain previous information and continously train the network on it and it works. Of course it's slower because you don't process all the new information, you mix it with old information. I guess there are better ways to do it.

1

EDMismyO2 t1_izy6ydb wrote

A similar idea is used with experience replay in DQNs. For RL, it's important to ensure failure states are retained in the replay buffer so it keeps being reminded they are a failure or it starts to forget and then does dumb things. In RL the phenomenon is called 'catastrophic forgetting'.

1

ChuckSeven t1_j016zp2 wrote

Doubt. I know the old stories too but large language models are essentially trained like that. Most never do an epoch twice and evaluate the model periodically.

1