[P] Built an at-cost, pay per second, open-source API for Tortoise text-to-speech (best I've heard!)

Submitted by Apprehensive-Tax-214 t3_107u88d in MachineLearning

Improve Tortoise TTS by 30% inference speed, and packaged it up as a hosted API that charges per-second. All code is open-sourced: https://github.com/metavoicexyz/tortoise-tts-modal-api, https://github.com/metavoicexyz/tortoise-tts

It can be used via a UI on: https://tts.themetavoice.xyz

There are more details here: https://twitter.com/vatsal_aggarwal/status/1612536547248836608?s=20

17

Comments

You must log in or register to comment.

Dragonsareforreal t1_j3oze0w wrote

Forgive me for my ignorance but isn’t TTTS opensource?

10

Impossible-Bus-6729 t1_j3rl5zi wrote

You are absolutely right. However, despite being open source, it requires a lot of time/effort/resources to get the tortoise text-to-speech synthesis working. Therefore, it's very difficult for a non-technical person to use, and for most developers looking to build applications on top it's a waste of time since they will have to reinvent the wheel to create an ML endpoint.

That is why we built this managed API on top for anyone to get started with testing the model / integrating TTTS quickly. Hope that helps?

We open-sourced our ML infrastructure for TTTS deployment so that others can contribute to make it faster, scale better or add features like easy fine-tuning with data.

3

iaalaughlin t1_j3qcffy wrote

30% faster than what?

How much does it cost? I’m not signing in just to see the pricing.

Why not use replicate’s api?

$0.00055 per second or $0.033 per minute. Is that cheaper or more expensive than what you are charging?

https://replicate.com/afiaka87/tortoise-tts

8

Impossible-Bus-6729 t1_j3rm2wg wrote

30% faster than the original TTTS!

We are providing the API at cost, i.e. $0.0005833/second.
We also provide $5 of free credits to try out the API. It currently costs $0.03/30words. Therefore, $5 ~ 160 API calls.

Based on your comment, its at par with replicate pricing. But, we will be faster with inference times & hence yielding micro-savings compounding over time

3

RuairiSpain t1_j3tj6zs wrote

Tried it, returns blank audio on my mobile

1

Apprehensive-Tax-214 OP t1_j4e6nex wrote

Can you try on laptop? Works on mobile for me, might just be an error on a system we've not tested...

1

MrHumun t1_j3x4s1p wrote

Where are you hosting GPU for this?

1

Apprehensive-Tax-214 OP t1_j4e6q51 wrote

We're using Modal who run Kubernetes-type GPU cluster on GCP.

1

MrHumun t1_j4f8i6q wrote

Ah, so you guys are 24x7 running a GPU server? isn't cost too high in this way? What you do when things are stale?

1

Apprehensive-Tax-214 OP t1_j4ftp26 wrote

Nope, a new container is spun up when someone sends a request and spun down when the request is over. This is why we're able to provide at-cost and charge per-second.

1

Unlikely-Advice-7168 t1_j4p515n wrote

Site doesn't work

1

Apprehensive-Tax-214 OP t1_j4qmjlc wrote

do you have a verified github email?

1

Unlikely-Advice-7168 t1_j4soi7h wrote

yup, I'm going to try it on another computer to see if it works there.
If it's just me then there must be a problem on my end

1

Apprehensive-Tax-214 OP t1_j4soopu wrote

So only two other people have had this error. One was resolved by verifying their GitHub email. Haven't heard back from the other.

1

Apprehensive-Tax-214 OP t1_j4qmlju wrote

do you have a verified github email?

1

Unlikely-Advice-7168 t1_j52rc7s wrote

For clarity, I've tried it with Chrome, brave, firefox on mobile and 2 laptops with two different github accounts. One I've used in the past and a new one I just made to test it.

1