disgruntled_pie t1_jc4ffo1 wrote
Reply to comment by atlast_a_redditor in [R] Stanford-Alpaca 7B model (an instruction tuned version of LLaMA) performs as well as text-davinci-003 by dojoteef
I’ve successfully run the 13B parameter version of Llama on my 2080TI (11GB of VRAM) in 4-bit mode and performance was pretty good.
pilibitti t1_jc56vv5 wrote
hey do you have a link for how one might set this up?
disgruntled_pie t1_jc5g6or wrote
I’m using this project: https://github.com/oobabooga/text-generation-webui
The project’s Github wiki has a page on llama that explains everything you need.
pdaddyo t1_jc5uoly wrote
And if you get stuck check out /r/oobabooga
sneakpeekbot t1_jc5upgp wrote
Here's a sneak peek of /r/Oobabooga using the top posts of all time!
#1: The new streaming algorithm has been merged. It's a lot faster! | 6 comments
#2: Text streaming will become 1000000x faster tomorrow
#3: LLaMA tutorial (including 4-bit mode) | 10 comments
^^I'm ^^a ^^bot, ^^beep ^^boop ^^| ^^Downvote ^^to ^^remove ^^| ^^Contact ^^| ^^Info ^^| ^^Opt-out ^^| ^^GitHub
pilibitti t1_jc5was5 wrote
thank you!
Viewing a single comment thread. View all comments