andreichiffa t1_j8hawd4 wrote on February 14, 2023 at 8:23 AM

Reply to comment by ateqio in [D] Looking for recommendations for an affordable API service to classify AI-generated text by ateqio

I have reported to Huggingface what its detector was used for and its failure modes (hint:false positives are worse). In the first days of December. They decided to keep it up. It’s on their consciousness.

Same thing with API providers. Those willing to sell you one are selling you snake oil. It’s on their consciousness.

Same thing for you. You want to build an app that sells snake oil that can be harmful in a lot of scenarios? It’s on your consciousness.

But at that point you even don’t need an API to build it.

ateqio OP t1_j8hcsz6 wrote on February 14, 2023 at 8:51 AM

What's the ratio of false positives? honestly curious

andreichiffa t1_j8hf2th wrote on February 14, 2023 at 9:24 AM

10% is what OpenAI considered as "good enough" for theirs, but the problem is with the fact that the detection is not uniform. Most neurodivergent folks will be misclassified as generative models, just as for people with social anxiety who tend to be wordy. Non-native and non-fluent English speakers are the other big false-positive triggers.