[Research] Can we possibly get access to large language models (PaLM 540B, etc) like GPT-3 but no cost?

Submitted by NLP2829 t3_yu8nna in MachineLearning

(I only want to do inference, I don't need to finetune it.)

I want to use very-large language model (#parameters > 100B) to do some experiments, is that true the only very-large language model we can get access to is GPT3 API? Can we possibly get access to PaLM and Flan-PaLM 540B with no cost by chance?

I have searched over the internet but can't find a definite answer. As GPT-3 pricing for text-davinci-2 is not cheap, I am wondering if there's a chance to use other models.

Also, I can request up to 372GB VRAM, is there any large language model (#parameters > 100B) that I can actually download and run "locally"?

37

Comments

You must log in or register to comment.

allwordsaremadeup t1_iw83d9j wrote

Bloom not big enough for you? 176B parameters. Can be downloaded here: https://huggingface.co/bigscience/bloom

30

learn-deeply t1_iwakal2 wrote

Bloom-175B is worse than NeoX-20B from empircal testing. Use NeoX and save on resources or OPT-175

17

ML4Bratwurst t1_iwaqxyc wrote

Do you really need such a big transformer? There a a couple of big transformers available in the Huggins face API.

2

Gnabenmeister t1_iwb3l2s wrote

I think metas was also published?

1

csreid t1_iwlyxnz wrote

>Also, I can request up to 372GB VRAM, is there any large language model (#parameters > 100B) that I can actually download and run "locally"?

I've never done anything non-trivial with LLMs but even using 32 bit floats for 100B parameters should take 400 gigs of RAM, right?

1

SJ5125 t1_iwwfez9 wrote

Most would use bfloat16 for LLMs

1