Purplekeyboard t1_j2s8it2 wrote on January 3, 2023 at 4:27 PM

Bloom's not very good, pruned or not.

Taenk t1_j2sgndx wrote on January 3, 2023 at 5:19 PM

Compared to what? I have been playing with it for a little bit via Petals and it performs decently, although ChatGPT certainly sets a very high bar of success. Personally I think that it is a shame, that OpenAI gets exclusive access to the absolutely massive dataset of interacting with actual humans and models like BLOOM could certainly profit from having publically accessible interactions.

nutpeabutter t1_j2snx76 wrote on January 3, 2023 at 6:04 PM

From my personal interactions it just gave off this vibe that it was trained on websites, rather than the GPT-3 (both base and chat) models which felt much more natural. Something to do with having to learn too many languages?

C0hentheBarbarian t1_j2sl0n3 wrote on January 3, 2023 at 5:46 PM

What about BLOOMZ? Isn’t it fine tuned in a similar way to GPT-3? Instruction fine tuned?

yahma t1_j2ssc01 wrote on January 3, 2023 at 6:31 PM

I wasn't very impressed with BLOOMZ. Responses seem short and optimized for Q/A style output. Perhaps Zero-Shot and single-shot worked better than Bloom, but Bloom seemed to produce better output for stories or writing in general.

I was only able to test the 6B models though, so not sure how the 176B models compare.