Verify that the power is off (at the breaker is best). . Re-verify that the power is off with a voltmeter.

Once you are sure that the chances of getting zapped are as close to nil as you can get them, you can use insulated needle nose pliers to spin the old base out.