hazard02 t1_j46e13z wrote on January 13, 2023 at 2:12 PM

Reply to comment by chimp73 in [D] Bitter lesson 2.0? by Tea_Pearce

I think one counter-argument is that Andrew Ng has said that there are profitable opportunities that Google knows about but doesn't go after simply because they're too small to matter to Google (or Microsoft or any megacorp), even though those opportunities are large enough to support a "normal size" business.

From this view, it makes sense to "outsource" the fine-tuning to businesses that are buying the foundational models because why bother with a project that would "only" add a few million/year in revenue?

Additionally, if the fine-tuning data is very domain-specific or proprietary (e.g. your company's customer service chat logs for example) then the foundational model providers might literally not be able to do it.

Having said all this, I certainly expect a small industry of fine-tuning consultants/tooling/etc to grow over the coming years

Nowado t1_j46klvj wrote on January 13, 2023 at 2:58 PM

From this perspective you could say there are products that wouldn't make sense for Amazon to bother with. How's that working out.

hazard02 t1_j46mbb6 wrote on January 13, 2023 at 3:09 PM

Edit:
OK I had a snarky comment here, but instead I'd like to suggest that the business models are fundamentally different: Amazon sells products that they (mostly) don't produce, and offers a platform for third-party vendors. In contrast to something like OpenAI, they're an aggregator and an intermediary.

ThirdMover t1_j46t3fc wrote on January 13, 2023 at 3:53 PM

I think the point of the metaphor was Amazon stealing product ideas from third party vendors on their site and undercutting them. They know what sells better than anyone and can then just produce it.

If Google or OpenAI offers people the opportunity to finetune their foundation models they will know when something valuable comes out of it and simply replicate it then. There is close to zero institutional cost for them to do so.

That's a reason why I think all these startups that want to build business models around ChatGPT are insane: if you do it and it actually turns out to work OpenAI will just steal your lunch and you have no way of stopping that.

Nowado t1_j4723n6 wrote on January 13, 2023 at 4:50 PM

That was precisely the point.

Amazon started as a sales service and then moved to become platform. Once it was platform, everyone assumed that sales business was too small for them.

And then they started to cannibalize businesses using their platform.

GPT-5entient t1_j4s8q64 wrote on January 17, 2023 at 10:14 PM

>I think the point of the metaphor was Amazon stealing product ideas from third party vendors on their site and undercutting them. They know what sells better than anyone and can then just produce it.

In many cases they are probably just selling the same white label item outright, just slapping on "Amazon Basics"...

Phoneaccount25732 t1_j477kis wrote on January 13, 2023 at 5:23 PM

The reason Google doesn't bother is that they are aggressive about acquisitions. They're outsourcing the difficult risky work.