Viewing a single comment thread. View all comments

ZestyData t1_j45bgzi wrote

This concept already exists so there are plenty of resources (papers, etc) online to learn from.

However, current code generation models are huge and hefty, and take a lot of time & resources to build using our current 2023 technology. So it probably isn't a great idea to build a large code-gen language model from scratch.

However, to do a school project about Large Language Models (LLMs), which includes finetuning a pretrained model as well as doing a small model from scratch as a demonstration, would be cool!

1