[P] searchGPT - a bing-like LLM-based Grounded Search Engine (with Demo, github)Submitted by michaelthwan_ai t3_11vi82q in MachineLearning
BalorNG t1_jcy0trr wrote
Reply to comment by michaelthwan_ai in [P] searchGPT - a bing-like LLM-based Grounded Search Engine (with Demo, github) by michaelthwan_ai
Yea, I'm sure that compact-ish distilled, specialised models trained on high quality, multimodal data is the way to go.
What's interesting, once generative models get good enough to produce synthetic data that is OF HIGHER QUALITY than laion/common crawl/etc, it should improve model quality which should allow to generate better synthetic data... not exactly singularity, but certainly one aspect of it :)
michaelthwan_ai OP t1_jcy73od wrote
Your idea sounds like GAN - maybe one model will generate high-quality synthetic data and another one try to 'discriminate' it, then they may output an ultra-high quality one finally (for another model to eat). And an AI model community is formed to self-improve...
BalorNG t1_jcy7l5d wrote
Yea, in a way something like this was already done with LLAMA-Alpaca finetune - they used chatgpt to generate instuct finetune dataset, what, while far from pefrect, worked pretty damn well.
Viewing a single comment thread. View all comments