Viewing a single comment thread. View all comments

MysteryInc152 t1_j6okowf wrote

Reply to comment by Zetsu-Eiyu-O in [D] Generative Model FOr Facts Extraction by Zetsu-Eiyu-O

Not sure what you mean by penalize but say you wanted an LLM that wasn't instruction fine-tuned to translate between 2 languages it knows.

Your input would be

Language x: "text of language x"

Language y: "translated language x text"

You'd do this for a few examples. 2 or 3 should be good. Or even one depending on the task. Then finally

Language x: "text you want translated"

Language y: The model would translate the text and output here

All transformer generative LLMs work the same way with enough scale. GPT-2 (only 1.5b parameters) does not have the necessary scale.

1

Zetsu-Eiyu-O OP t1_j6oztfw wrote

Oh, I see thanks, I have a few questions about the basics of training a large language model, do you mind if I shoot you a message?

1