Viewing a single comment thread. View all comments

visarga t1_je6k74d wrote

Reply to comment by Im_Unlucky in [D] The best way to train an LLM on company data by jaxolingo

Often it seems that the model can't properly synthesise information from a bunch of snippets, it lacks the context of those snippets, so it will combine the information incorrectly or hallucinate an explanation.

Retrieval + loading data in the context is far from solved.

16

Im_Unlucky t1_je6vjqd wrote

This is true. Good prompt engineering has definitely gone a long way for me in improving rag performance though. I don’t think shoving snippets into the prompt is the long term solution, but at this point in time it’s the best option in my experience.

8

kromem t1_je6uv46 wrote

"Moar layers" doesn't only need to apply to the NN.

CoT prompting works by breaking analysis down into smaller steps that each generate their own additional context.

Doing something similar with DB analysis is absolutely possible, such as preemptively summarizing schema and using that summary as part of the retrieval to contextualize the specific fragments.

Additionally, having static analysis examples on hand for related tables that's fed in to go from zero shot to few shot would go a long way at reducing some of the issues you highlight.

6

Tostino t1_je847jg wrote

Literally just worked through this today manually as a proof of concept, using the LLM to augment the DB schema with comments describing any relevant info or corner cases. I'm essentially just manually feeding it as context to my prompts when I need to know something related to that set of tables, but it seems pretty powerful. Automating this is going to be nuts.

5

kromem t1_je84zam wrote

> Automating this is going to be nuts.

Yes, yes it is.

6