I'd take a drill over a driver.

I'd rather have the chuck that can hold just about anything, than deal with a driver that can only hold certain kinds of bits. I feel like it gives you a bit more flexibility in what you can do. Sooner or later, you're going to need a holesaw or something, and you're going to find yourself saying, "Shit, I should've bought the drill"

Or just get both. They usually come in sets nowadays anyway. It's very convenient to have both.