Llukas88 t1_jdimn2w wrote on March 24, 2023 at 5:34 PM

Reply to comment by to4life4 in [D] What is the best open source chatbot AI to do transfer learning on? by to4life4

The Alpaca model based on LLaMa isnt. The dataset, which is also called Alpaca is. If you train Bloom, which uses a permissive license, on this dataset, the Bloom license is applied to your finetuned model and you should be able to use it commercially.

to4life4 OP t1_jdin0kc wrote on March 24, 2023 at 5:37 PM

Ah ok cool gotcha. Any benchmarks on Bloom performance vs Alpaca and others?

Llukas88 t1_jdiohwe wrote on March 24, 2023 at 5:46 PM

Not any i know of, played around today with Alphacoom (https://huggingface.co/mrm8488/Alpacoom) and got pretty Bad results then tried a BloomZ version (https://huggingface.co/mrm8488/bloomz-7b1-mt-ft-alpaca) and got results similar to the Alpaca-Native model. Maybe read the BloomZ paper it should be a pretty good basis to build a chat model, rest should depend on your Training approach and Data.