ShadowStormDrift t1_iu3fkqs wrote on October 28, 2022 at 7:26 AM

I code up a semantic search engine. I was able to get it down to 3 seconds for one search.

That's blazingly fast by my standard (used to take 45 minutes) that still haunts my dreams. If 10 people use the site simultaneously that's 30 seconds before number 10 gets his results back. Which is unacceptable.

So yes. I do care if I can get that done quicker.

GPUaccelerated OP t1_iu4umuw wrote on October 28, 2022 at 3:45 PM

Yeah, see in your use case, speed makes so much sense. Thank you for sharing.

Mind sharing that site with us here?

I'm always interested in taking a look at cool projects.

Also what kind of hardware is currently tasked with your project's inference?

ShadowStormDrift t1_iu53ih6 wrote on October 28, 2022 at 4:44 PM

Of course!

www.sasdghub.up.ac.za

The semantic search as well as a few other key features haven't made it up yet. We're aiming to have them up end of November, mid December.

We've got a two server setup with the second being our "Work-horse" intended for GPU related jobs. It's an RTX 3090 with 32GB VRAM, 64GB DDR4 RAM and a 8 core CPU (I forget it's exact setup)

GPUaccelerated OP t1_iuim3cp wrote on October 31, 2022 at 4:42 PM

Very cool! But I think you mean 24GB of VRAM for the 3090?

Issues loading the web page, btw.

ShadowStormDrift t1_iuivphh wrote on October 31, 2022 at 5:45 PM

https://sasdghub.up.ac.za/en/

Let's try this lol.

GPUaccelerated OP t1_iuixkzj wrote on October 31, 2022 at 5:57 PM

So cool! Good for you!