"Sounds different" is the answer you're going to get from headphone users (this sub), if you really want to know why they sound different, you need to ask a headphone designer / engineer.

And I don't think that's a huge population of people, so you'd be quite fortunate to find one on this sub who knows the answer and is going to take the time to write an answer you find satisfactory.

My impression, as someone who is not a headphone engineer, is that there are an insane number of variables that go into how sound is perceived through a headphone ("psychoacoustics"), and the way that different headphones design around those variables is complicated. There's certainly more to it than the size of the driver. Like one of the interesting things about the Focal Elegia closed back is that they have sound baffling inside the earcup that dissipates low frequencies, to reduce resonance, and it seems to make them sound more like an open-back headphone. I think another variable is how consistent the left and right ears are in their frequency response, as inconsistency can muddy up the directional imaging. And that's something that depends on really tight controls in the manufacturing process with all the different components, and the way in which they're assembled.

That's just a flavour I've what I've been able to deduce from being around the hobby, I don't really know the answer. I just try out different headphones, and some of them sound better to me than others (in ways that aren't totally captured by frequency response, since I typically EQ all my headphones to the Harman curve)