I think you can do it both ways (with or without filtering) and compare. Just speculating now, but the filtering could potentially affect the performance on the other classes (since you change the model’s training path for those classes ). But my guess is that that should not be a big issue, so I’d probably go about it the way you described if I had to pick one strategy.