Viewing a single comment thread. View all comments

MysteryInc152 t1_jcdthob wrote

Are you using Gpt-Vision ? Or are there separate assortments of visual foundation models ?

2

Empty-Revolution7570 OP t1_jcdtuff wrote

Yes, I included all the VFMs. I added upon those a few more, such as OpenAI Whisper. Still exploring how to incorporate video models

1

MysteryInc152 t1_jcduvhn wrote

I'm sorry maybe I want clear but you obviously have API access to GPT-4 right ? Does this access include an API call to their Vision model ? Or are you sending the images straight to BLIP and the like.

2

Empty-Revolution7570 OP t1_jcdv1nt wrote

No, it understands image through other models on hugging face, and outputs image with diffusers or OpenAI dalle

1