Check out Boggleheads and the 3-fund portfolio for what specifically to invest in.

Personally, being in a very similar spot to you, we are not paying early. Instead, we put as much as we are comfortable with into the market and the rest into T-bills and I-bonds.

Also, note that you are comparing your 4% return at 6 years to the total mortgage term (30 years?). You clearly make more money at 4% rates than at 3% costs. The issue is that the 4% isn't guaranteed and the 3% is.

As well, if you have to borrow to pay something, it's hurting someone else who could have gotten me for their money, but they gave it to you instead. So if you want to borrow to pay it off, at least give them the rates they could have earned by putting it into a HYSA or T-bill, etc instead.

Finally, don't forget that insurance and taxes still get paid when the mortgage is done.