[P] SimpleAI : A self-hosted alternative to OpenAI API

Submitted by lhenault t3_122tddh in MachineLearning

Hey everyone,

I wanted to share with you SimpleAI, a self-hosted alternative to OpenAI API.

The aim of this project is to replicate the (main) endpoints of OpenAI API, and to let you easily and quickly plug in any new model. It basically allows you to deploy your custom model wherever you want and easily, while minimizing the amount of changes both on server and client sides.

It's compatible with the OpenAI client so you don't have to change much in your existing code (or can use it to easily query your API).

Wether you like or not the AI-as-a-service approach of OpenAI, I think that project could be of interest to many. Even if you are fully satisfied with a paid API, you might be interested in this if:

  • You need a model fine tuned on some specific language and don't see any good alternative, or your company data is too sensitive to send it to an external service

  • You’ve developped your own awesome model, and want a drop-in replacement to switch to yours, to be able to A/B test the two approaches.

  • You're deploying your services in an infrastructure with an unreliable internet connection, so you would rather have your service locally

  • You're just another AI enthusiast with a lot of spare time and free GPU

I've personally really enjoyed how open the ML(Ops) community has been in the past years, and seeing how the industry seems to be moving towards paid API and black box systems can be a bit worrying. This project might be useful to expose great, community-based alternatives.

If that sounds interesting, please have a look at the examples. I also have a blogpost explaining a few more things.

Thank you!

123

Comments

You must log in or register to comment.

HatsusenoRin t1_jdtanqn wrote

Nice work. Do you have (or have suggestions for) a simple web frontend for testing (instead of lots of curl commands)?

8

ryanjkelly2 t1_jdtj5vw wrote

You could use Postman.

4

HatsusenoRin t1_jdty1by wrote

Thanks. However I don't think it's made specifically for this API and it forces me to open new account to use it (usually the app goes to my trash bin as soon as I realize it).

I'm surprised that there's no open-source frontend for testing this popular API.

−1

C4ptainK1ng t1_jdvqoy2 wrote

You dont need to create an Account for postman. You can just skip the registration

1

Beli_Mawrr t1_je394op wrote

Try Jquery AJAX. that works on the frontend. $.post, $.get, etc.

1

lhenault OP t1_jdukg00 wrote

Hey thank you for the feedback! As r/ryanjkelly2 suggested, you could indeed use Postman, but I believe the easiest way is to use the already included Swagger UI, available at <base_url>/docs.

If your goal is to have a slightly more friendly UI for end users, it should be relatively easy to build something custom, using the OpenAI clients (or requests package) and something like Streamlit. Or even a notebook (you can use the OpenAI cookbook as a starting point).

3

HatsusenoRin t1_jduyliz wrote

Thanks for your help. I think I'll take a look at the /docs.

3

onequark t1_jdx5o96 wrote

Awesome, does it support token streaming?

3

lhenault OP t1_jdxpuso wrote

Not for now but that’s indeed a cool feature and something available in OpenAI API. It shouldn’t be too hard to implement, as I’ve already started something for that on the gRPC backend, and as FastAPI has a StreamingResponse. Thanks for suggesting it, will try to prioritise this!

2