Ok, well, sit down and think things through and get an answer that is more definitive than "I'm assuming". This isn't the time for ambiguity.

Then, when you talk, explain to him why you said that. Apologize again sincerely once, but don't belabor it. Reassure him that you don't feel that way, and use specific examples to back it up.

After that, you've really done all you can do. I don't think you should bend over backwards too much for a comment like this.