Viewing a single comment thread. View all comments

race2tb t1_jcops9d wrote on March 18, 2023 at 11:51 AM

I think there is a need for a p2p torrent backend for models that needs to exist now.

marcus_hk t1_jcrdufd wrote on March 18, 2023 at 11:46 PM

For weights, yes, and for inference. If you can decompose and distribute a model across enough nodes, then you can get meaningful compute out of CPUs too — for instance for tokenization and smaller models.