Viewing a single comment thread. View all comments

amassivek t1_izoh41k wrote

There is a framework for learning with forward passes, a friendly and thorough tutorial: https://amassivek.github.io/sigprop .

The most interesting insights from the framework:

  • This algorithm provides an explanation for how neurons in the brain without error connections receive learning signals.
  • It works for continuous networks with hebbian learning. This provides evidence for this algorithm as model of learning in the brain.
  • It works for spiking neural networks using only the membrane potential (aka voltage in hardware). This supports applying this algorithm for learning on neuromorphic chips.

The Signal Propagation framework paper: https://arxiv.org/abs/2204.01723 . The Forward-Forward algorithm is an implementation of this framework.

I am an author of this work. I was presenting this work at a reading group when one of the members pointed out the connection between signal propagation and forward forward.

4