Viewing a single comment thread. View all comments

nail_nail t1_j9gkc5w wrote

Reply to comment by hayAbhay in Fine tuning a GPT for text generation by nashcaps2724

That means it will need to be paid for each summarization API call, forever, right? Is there an alternative model that one can tune on a couple of high end nvidia cards? Like GPT NeoX?

1

hayAbhay t1_j9i9nfv wrote

If you have the hardware, and if you have a lot of those input-output examples, you can use alternative smaller models in the gpt family.

Should work reasonably well especially if the variance in the input-output isn't too much. (A lot depends on your dataset here)

Definitely tradeoffs here in terms of model dev, inference and maintenance of it. If the expected costs aren't too high, I'd strongly recommend gpt3 as a base.

1