Viewing a single comment thread. View all comments

cfoster0 t1_izlys6v wrote

About this bit

> At the moment, TRLX has an API capable of production-ready RLHF at the scales required for LLM deployment (e.g. 33 billion parameters). Future versions of TRLX will allow for language models up to 200B parameters. As such, interfacing with TRLX is optimized for machine learning engineers with experience at this scale.

Has TRLX been used to tune models in production already? Or if not, what did the blog post mean by "capable of production-ready RLHF"? I haven't seen any RLHF-ed models built on open source software yet, much less a 33B parameter one.

EDIT: Also hi @FerretDude

9

FerretDude t1_izoa26g wrote

It's already being used in production with a number of our partners. We have some chonky models coming out really soon. Expect things well into the tens of billions in the coming months.

3

cfoster0 t1_izrdeii wrote

Who? Who's even using RLHF in production yet, besides OpenAI (and maybe Cohere)?

5

FerretDude t1_izs8wj1 wrote

Not allowed to share, many groups are looking into using RLHF in production though

−1

cfoster0 t1_izuxn52 wrote

Did y'all stop doing work out in the open? That's a shame. End of an era, I guess.

2

FerretDude t1_izyu3ka wrote

RLHF is a bit tricky because you have to either work with data vendors or groups that have access to feedback data. Eventually we'll rely more on crowd sourcing I think.

2