I have some examples / benchmarks here: https://huggingface.co/datasets?other=speechbox_punc

Haven't done a in-depth test yet, do you have some links to other existing codebases? :-)

​

Here are some tests: https://huggingface.co/datasets?other=speechbox_punc

The way it's implemented, Whisper cannot hallucinate because it can only predict letters of the original normalized transcript or punctuation, so the algorithm in speechbox guarantees that Whisper cannot hallucinate (you can think of it as a very restricted beam search)