You're just missing out, that's for sure. Michael Imperioli is good (as usual), but he is not carrying the show by any means. I stand by the fact that you have to enjoy this show in chronological order. It plays with your expectations and while the storyarchs are self contained, there are clear parallels and call backs. Also, with one of the characters you're missing half of the story if you skip season 1.

It's a quick watch anyway. 8 episodes if I'm not mistaken.