Llukas88 t1_jdilkns wrote on March 24, 2023 at 5:28 PM

There are Alpaca finetuned versions of Bloom or BloomZ on huggingface, maybe try those. Another option would be the Chat version of GPTNeoX from OpenChatKit. Both should be Open Source and free to use.

to4life4 OP t1_jdim3ip wrote on March 24, 2023 at 5:31 PM

Someone said that Alpaca isn't open source though?

Llukas88 t1_jdimn2w wrote on March 24, 2023 at 5:34 PM

The Alpaca model based on LLaMa isnt. The dataset, which is also called Alpaca is. If you train Bloom, which uses a permissive license, on this dataset, the Bloom license is applied to your finetuned model and you should be able to use it commercially.

to4life4 OP t1_jdin0kc wrote on March 24, 2023 at 5:37 PM

Ah ok cool gotcha. Any benchmarks on Bloom performance vs Alpaca and others?

Llukas88 t1_jdiohwe wrote on March 24, 2023 at 5:46 PM

Not any i know of, played around today with Alphacoom (https://huggingface.co/mrm8488/Alpacoom) and got pretty Bad results then tried a BloomZ version (https://huggingface.co/mrm8488/bloomz-7b1-mt-ft-alpaca) and got results similar to the Alpaca-Native model. Maybe read the BloomZ paper it should be a pretty good basis to build a chat model, rest should depend on your Training approach and Data.