Submitted by invertedpassion t3_yakn9y in MachineLearning
Like RETRO or something similar.
Does anyone know if any retrieval based language model is available to download?
Submitted by invertedpassion t3_yakn9y in MachineLearning
Like RETRO or something similar.
Does anyone know if any retrieval based language model is available to download?
RetroMAE is available on the HuggingFace hub: https://huggingface.co/Shitao
RetoMaton: https://github.com/neulab/retomaton
I can't vouch for it personally--and it may or may not meet your needs (noting that this is about repurposing existing non-retrieval models to meet a maybe-similar need)--but maybe check out: https://twitter.com/bencmejla/status/1583499775789654016
There's a Github project that an individual put together based on the RETRO paper. If you checkout the issues list there is some info on work on a pretrained model.
There is also the Huggingface RAG Model and Facebook has a couple of models on the HF hub.
Note that the RAG model is an an older approach to retrieval so you probably want to be looking at the RETRO project above.
I’ve been following that repo - no pretrained yet
Yes, which ones?
Try looking up DensePhrases, it was made by a colleague of mine and may be what you're looking for. They also have an online demo you can try.
I'm not sure what you mean by "retrieval-based language model" though. I don't think there's any language model that's made solely for the purpose of retrieval.
RETRO by Google was made for that, no?
howard__wolowitz t1_itbmoax wrote
You may look into RetGen by Microsoft. The repo also has model checkpoints linked too for download.