Jaffa6 t1_jd03par wrote on March 20, 2023 at 9:28 PM

Reply to comment by Haghiri75 in Alpaca-7B and Dalai, how can I get coherent results? by Haghiri75

That's odd.

Quantisation should make it go from (e.g.) 32 bit floats to 16bit floats, but I wouldn't expect it to lose that much coherency at all. Did they say somewhere that that's why?

Haghiri75 OP t1_jd32s29 wrote on March 21, 2023 at 2:18 PM

Apparently I was wrong, the problem is not only quantization. It is because it's not Stanford's Alpaca and another alpaca-like model. This was what I can surely say about that.