Viewing a single comment thread. View all comments

starstruckmon t1_j3wxccu wrote

Reply to comment by yaosio in [D] Microsoft ChatGPT investment isn't about Bing but about Cortana by fintechSGNYC

True and that's probably the reason. But still, they have a ML/AI division. Why not have them just train Megatron to convergence and leapfrog GPT3? I'll never understand how these companies make decisions honestly.

1

m98789 t1_j3x653d wrote

The three main AI innovation ingredients are: talent, data, and compute. Microsoft has all three, but of them all, at the world-class level, top talent is the most scarce. Microsoft has amazing talent in MSR but it is spread into multiple areas and has different agendas. OpenAI talent is probably near/on par with MSR talent, but has focus and experience and a dream team dedicated to world-class generative AI. They will be collaborating with MSR researchers too, and leveraging the immense compute and data resources at Microsoft.

3

erelim t1_j3x3pl1 wrote

Everyone is currently behind openAI even Google who likely considers this existential risk. If you were Google/MS would you rather buy and become the leader and their talent or let the competitor buy them, thinking you can build something from behind to overtake the leader. The latter is possible but riskier than the first

1

starstruckmon t1_j3x4bsr wrote

How is Google behind OpenAI? Chinchilla has simmilar performance as GPT3 yet is much cheaper to run since it has less than half the parameters.

2

visarga t1_j41avq0 wrote

Many smaller models give good results on classification and extractive tasks. But when they need to get creative they don't sound so great. I don't know if Chinchilla is as creative as the latest from OpenAI, but my gut feeling says it isn't.

1

starstruckmon t1_j41dgsk wrote

There's no way for us to tell for certain, but since Google has used it for creativity oriented projects/papers like Dramatron, I don't think so. I feel the researchers would have said something instead of leading the whole world intentionally astray as everyone is now following Chinchilla's scaling laws.

Chinchilla isn't just a smaller model. It's adequately trained unlike GPT3 which is severely undertrained, so simmilar, if not exceeding ( as officially claimed ), capabilities isn't unexpected.

1