You didn't lose $800 in interest, you paid $800 to drive around a car you didn't own (all of, yet). Sure, lower is better, but you have think about interest in terms of what it gave you. At 4.99%, you made use of about $16,000 worth of something you didn't own.

Meanwhile, sure, your market investments probably aren't doing that well, but you're buying shares on sale that will pay off later. And the money you've kept as cash as an emergency fund isn't wasted, either, because you didn't know for sure that you wouldn't get laid off, just like you don't know for sure right now that you won't get laid off this year, so it has value as risk mitigation even just sitting in a HYSA. So when you spent $800 for the privilege of using a car you hadn't bought yet, it allowed you to put money into these other things that will help you as much or more than that $800 would have.

4.99% is right on the edge where paying off early for guaranteed return might make sense I think, so sure, make extra payments on it if you feel like it, because it's a safe-ish way to get a good return on a portion of your money, but don't think about the absolute dollar amount of interest as a waste without putting in context of what it bought you.