I mean, audio is just the type of data, it is still represented as an ordered series of points. If i remember on the timeseriesclassification.com website you got quite a lot of audio datasets.

For models, you could look at libraires like sktime, convst, tslearn.

If you don't care about speed or interpretability, I would suggest looking at HIVE COTE 2. If you need faster training, ROCKET or RDST/RDST ensemble (in convst), or simply a 1-NN with DTW, which can represent a baseline.