It's annoying because if you basically tap anywhere on the screen the big album art goes away. So that's why it's very easy to make it smaller. But you'll almost never accidentally make it bigger because expanding it requires you to click on the small album art, which is more deliberate.