Viewing a single comment thread. View all comments

GoofAckYoorsElf t1_j3ilywu wrote

This is all great. The only problem is that I can't use it due to non-disclosure and IP protection of my employer. As long as I have to send code over the web, it's a no-no.

13

IshKebab t1_j3j1gkz wrote

Yeah I imagine that will be an issue for lots of people. What's the SotA in open source LLMs?

I looked it up. Apparently it's BLOOM. Slightly bigger than GPT-3. No idea if it is better.

You need a DGX A100 to run it (only $150k!).

12

LetterRip t1_j3n91mt wrote

I'd do GLM-130B

> With INT4 quantization, the hardware requirements can further be reduced to a single server with 4 * RTX 3090 (24G) with almost no performance degradation.

https://github.com/THUDM/GLM-130B

I'd also look into pruning/distillation and you could probably shrink the model by about half again.

2

--algo t1_j3kxv2l wrote

How do you deal with source code hosting?

2

GoofAckYoorsElf t1_j3l2sls wrote

A cloud hosted GitLab with customer managed keys. We have a very detailed IP and security agreement with our cloud provider.

2

keepthepace t1_j3o8avv wrote

I am willing to be that 99% of the code is overprotected and no one in OpenAI would spend valuable time looking at it.

These protections mostly exist to justify some bullshit jobs within the company.

1

GoofAckYoorsElf t1_j3po3ti wrote

Probably. I'm still getting fired if I do something like that without permission.

2