I think he was mixing up resentment for...whatever the hell he feels toward Fiona. Which might have resentment thrown in...but it's also a helluva lot darker than that. And because they grew up in a similar upbringing, he's assuming that Cas must have the same feelings he does.

It's sorta like how racists think everyone around them will agree with their racist, for a (somehow!) less extreme example...