Viewing a single comment thread. View all comments

rowleboat t1_jctpu8c wrote

Reply to comment by michaelthwan_ai in [P] searchGPT - a bing-like LLM-based Grounded Search Engine (with Demo, github) by michaelthwan_ai

Can this use a SQL database as an external reference?

9

Tostino t1_jctq5az wrote

Look into llama-index

14

michaelthwan_ai OP t1_jctv2tm wrote

Thank you.
Due to people close to me and my googling, my choices of indexer is like this

pyterrier -> faiss -> native embedding

Then I found llama-index, but it currently won't give extra values to me so I didn't adopt.

I have stories on pros/cons on those lib...

11

michaelthwan_ai OP t1_jctvcas wrote

Theoretically yes but in exact the objective you want to do is crucial.

SQL database don't support similarity/elastic search, which is very useful in natural language. It may limit what you can do or make your product less good.

5

Secret-Fox-5238 t1_jcv5dhh wrote

This is completely false. Elastic was invented by SQL. You use things like “LIKE” and a few other choice keywords. Just google them or go to Microsoft directly and look at sql select statements. You can string together CTE’s which immediately gives you elasticity. So, sorry, but this is a nonsensical response

−5

michaelthwan_ai OP t1_jcws6h8 wrote

ChatGPT said what I want to say.

>I apologize for any confusion or misinformation in my previous response. You are correct that SQL databases do support various text search and similarity matching features, including the use of keywords like LIKE and CTE (Common Table Expressions) to enable more flexible and efficient querying.
>
>While it's true that specialized tools like Elasticsearch, Solr, or Algolia may offer additional features and performance benefits for certain natural language processing tasks, SQL databases can still be a powerful and effective tool for storing and querying structured and unstructured data, including text data.
>
>Thank you for bringing this to my attention and allowing me to clarify my previous response.

3