spiky_sugar
spiky_sugar t1_j3ley7v wrote
Reply to comment by learningmoreandmore in [D] I want to use GPT-J-6B for my story-writing project but I have a few questions about it. by learningmoreandmore
It depends. It really varies depending on what parameters you set for the generation. The choice of decoding and output text length can dramatically change the speed and quality of the outcome.
GPT-J-6B model I would say that it is possible to generate 10000 requests in few hours. Using only CPU will take much longer, but you could maybe generate 2000 requests in 24 hours. But again, it is strongly dependent on input and output text length and decoding type.
spiky_sugar t1_j3lbllf wrote
Reply to [D] I want to use GPT-J-6B for my story-writing project but I have a few questions about it. by learningmoreandmore
I am not sure what kind of stories you plan to generate, but I would recommend to look at huggingface model repositories. There are many models that have been already finetuned for the specific topic - for example https://huggingface.co/KoboldAI. It can save you lots of money and time.
spiky_sugar t1_je9phxc wrote
Reply to [D] Can large language models be applied to language translation? by matthkamis
https://arxiv.org/abs/2301.08745