Viewing a single comment thread. View all comments

CriticalTemperature1 t1_jdyubo2 wrote

Unfortunately the nature of this field is "the bitter lesson", scale trumps everything in machine learning so unfortunately/fortunately we are getting interested in language models when the scale is so large that it is impossible to make in impact on them unless you own your own $xxM company.

However, there are several interesting research avenues you can take:

  1. Improve small models with RLHF + fast implementations for a specific task (e.g. llama.cpp)
  2. Chaining models together with APIs to solve a real human problem
  3. Adding multimodal inputs to smaller LLMs
  4. Building platforms to make it easy to train and serve LLMs for many use cases
  5. Analyzing prompts and understanding how to make the most of the biggest LLMs
25

visarga t1_jdztq3o wrote

In short, build around LLMs and with LLMs, but don't compete directly with them.

4

SlowThePath t1_je2d9oi wrote

Yeah I don't see any startup being able to acquire the resources and time to catch up let alone compete or surpass. Unless they come up with some very novel new magic secret sauce which seems extremely unlikely.

1