hazard02

hazard02 t1_j46mbb6 wrote

Reply to comment by Nowado in [D] Bitter lesson 2.0? by Tea_Pearce

Edit:
OK I had a snarky comment here, but instead I'd like to suggest that the business models are fundamentally different: Amazon sells products that they (mostly) don't produce, and offers a platform for third-party vendors. In contrast to something like OpenAI, they're an aggregator and an intermediary.

10

hazard02 t1_j46e13z wrote

Reply to comment by chimp73 in [D] Bitter lesson 2.0? by Tea_Pearce

I think one counter-argument is that Andrew Ng has said that there are profitable opportunities that Google knows about but doesn't go after simply because they're too small to matter to Google (or Microsoft or any megacorp), even though those opportunities are large enough to support a "normal size" business.

From this view, it makes sense to "outsource" the fine-tuning to businesses that are buying the foundational models because why bother with a project that would "only" add a few million/year in revenue?

Additionally, if the fine-tuning data is very domain-specific or proprietary (e.g. your company's customer service chat logs for example) then the foundational model providers might literally not be able to do it.

​

Having said all this, I certainly expect a small industry of fine-tuning consultants/tooling/etc to grow over the coming years

26