Viewing a single comment thread. View all comments

yahma t1_jd24u9z wrote

Would switching the base pythia-12b model for llama-13b improve things?

2

RedditLovingSun t1_jd3nidx wrote

I don't think they can use llama cause of the limited open source rule fb put in llama. Wouldn't be as entirely open as pythia

3

yahma t1_jd6ptt5 wrote

Good point, I forgot about this.

1

timedacorn369 t1_jd25c76 wrote

Yeah I hope so. Compared to other chat assistants this seems to have a much better rlhf part but somewhat bad text completion based on my limited analysis of using them . So if they use LlaMa I think the output would substantially improve .

1