Viewing a single comment thread. View all comments

juliensalinas OP t1_jd2owfz wrote

Reply to comment by No_Combination_6429 in [D] An Instruct Version Of GPT-J Using Stanford Alpaca's Dataset by juliensalinas

Sure. Here's the repo I used for the fine-tuning: https://github.com/kingoflolz/mesh-transformer-jax. I used 5 epochs, and appart from that I kept the default parameters in the repo.

I haven't tried the LoRa approach yet. Do you think it could improve quality?

1