gdiamos t1_j4a96pu wrote on January 14, 2023 at 6:42 AM Reply to comment by mugbrushteeth in [D] Bitter lesson 2.0? by Tea_Pearce Currently we have exascale computers, e.g. 1e18 flops at around 50e6 watts. The power output of the sun is about 4e26 watts. That's 20 orders of magnitude on the table. This paper claims that energy of computation can theoretically be reduced by another 22 orders of magnitude. https://arxiv.org/pdf/quant-ph/9908043.pdf So physics (our current understanding) seems to allow at least 42 orders of magnitude bigger (computationally) learning machines than current generation foundation models, without leaving this solar system, and without converting mass into energy... Permalink Parent 1
gdiamos t1_j4a96pu wrote
Reply to comment by mugbrushteeth in [D] Bitter lesson 2.0? by Tea_Pearce
Currently we have exascale computers, e.g. 1e18 flops at around 50e6 watts.
The power output of the sun is about 4e26 watts. That's 20 orders of magnitude on the table.
This paper claims that energy of computation can theoretically be reduced by another 22 orders of magnitude. https://arxiv.org/pdf/quant-ph/9908043.pdf
So physics (our current understanding) seems to allow at least 42 orders of magnitude bigger (computationally) learning machines than current generation foundation models, without leaving this solar system, and without converting mass into energy...