Yes, but you're working with older models. But the advantage is that you can adapt them to you. You can get results that you can't get with all the online models. But they lag behind from the current modulation. And I just wonder is it worth it? By the time I have downloaded the whole thing, trained and get results, the whole thing is already 2-3 versions further. I'm running behind on quality.

LLaMA 13B/30B and LLaMA 7B with the Alpaca LoRA are the best that can be run locally on consumer hardware. LLaMA 65B exists but I wouldn't count that as something that can be run locally by most people.

From my own testing, the 7B model with the LoRA is comparable to 13B in coherency, and it's generally better than the recently released OpenAssistant model. If you'd like to see some examples, I answered many prompts in a r/singularity AMA for Alpaca. Go to this post and sort by new to see the responses. I continued where the OP left off.

Do you have specific chart between consumer hardware and performance of LLaMA 7B to 65B? Like I want to know if my poor gaming PC could have an response in under 1 minute?

I think that Cosmo-XL has to be one of the best dialogue-focused models available that is really lightweight

I’m having a hard time understanding what LoRA is and why it makes the 7B model better? I thought it only improves hardware requirements, but it also improves model coherency? This is all new for me

LoRA is how you train llama into alpaca on consumer hardware

Tom's Hardware has an article on that: https://www.tomshardware.com/news/running-your-own-chatbot-on-a-single-gpu

the LoRA is like a modular refinement of the base language model, in this case it's the part that makes it feel like a chatbot / assistant, and makes it follow instructions.

you can see the same concept over at civitai.com, filter by LoRAs. Something like a LoRA for one character can be run on different checkpoints that focus on photorealism or anime, etc.