Thanks. So bit confused here. Ot mentions needing an A100 to train. Am I able to run this off a 3090?

Couldnt it be possible to set up a large community Q/A repositiry then? Just crowdsource whatever it outputs and document collectively.

I was talking to GPT 4 about this and it said that it seems plausible and can dramatically bring down costs.

It called it "knowledge distillation"

It also mentioned that if we had access to the weights from open ai you can use a process called model compression to scale down the hardware and put it on less powerful gpus or distributed gpus (like how render farms work)

This also explains why open ai is so cagey on releasing weights - the initial training cost is where the money sink is but once weights are out their is ways to make it run on cheaper hardware.

But Im wondering does this mean the smaller model can ONLY respond to the questions your generating or will it have latent knowledge outside of just the knowledge transfer? Like would say the smaller model thats trained off this approach also be able to answer questions on topics that are "restricted" in open ais view that you couldnt ask it or do you absolutely must need to get an initial answer for such restricted content for it to be able to produce a responce?

Talking about things like writing malicious code or what not. I dont plan on doing that obviously but Im curious on if this means that these smaller models will basically be totally unrestricted now or if its just trained on say tons of python code it can just create said malicious code from scratch without actually being exposed with examples of "how" to make it (since it has a greater knowledge of the ubderlying principals of python)

Edit: Okay guess it can per GPT 4.

Damn these things are fascinating.

>Yes, the same concerns can apply to a smaller model being trained from a larger one via knowledge distillation. Knowledge distillation is a technique where the smaller model learns to mimic the larger model's behavior by training on a dataset generated using the larger model's outputs. The smaller model effectively learns from the larger model's knowledge and understanding of language patterns and concepts.

>As a result, the smaller model can also gain latent knowledge about various topics and domains, even if it hasn't been explicitly exposed to specific examples during training. This means that the smaller model could potentially generate undesirable content based on its understanding of the relationships between words and concepts, similar to the larger model.

Of course they dont. They're also wrong saying that men cant gain muscle while in a caloric deficit. Its harder mind you since you need to ensure muscle protein synthesis is higher than muscle protein breakdown by eating more protein than usual but it IS possible so I would take w.e. they said here with a grain of salt.

Wow. Had to look this up. A defector said the uncle was executed by firing squad of anti aircraft guns - another says he was ripped apart by 120 hungry dogs.

I guess we really dont know what goes on behind the scenes but that is some next level super-villain stuff.

The funds are already being moved.

https://www.reddit.com/r/CryptoCurrency/comments/zyruws/sam_bankman_just_cashed_out_600k_in_violation_of/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button

People can get put away for 15 years for holding more than 25 grams of pot in Florida.

Apparently literally trying to destroy democracy is less of an offence than drug laws.

Deja vu is just your brain trying to remember something - failing - and then just making it seem familiar anyways to cover its own ass.

Dont trust that guy.

Easier said than done after experiencing the whole consciousness thing lol.