Then you fundamentally misunderstand the implications of the phrase. To say that you literally "could care less" would mean anything from carrying a miniscule amount, all the way up to caring completely - it would become essentially meaningless. Just accept that you're wrong and stop pretending.