If your removing the tile it doesn't matter what it is doing or not doing. I would recommend a good solid surface/base under your new tile. Don't do spacing of your new tike like the existing. 1/4" is a good gap. The grout line along your tub base should be caulked with caulking the same color after grouting. Tubs are sometimes flexible at their base which isn't a kind environment for grout. The rust you speak of just might be mineral stain left behind from water getting out of the tub.

If grout is cracking it means tiles are loose. Floor needs to be taken up and redone. Use concrete board, set in thinset and screwed, as new base. Check for leaks from tub, toilet and vanity prior to installing new floor. Repair water leaks and other issues first. Good luck.

Grout in a change of plane (eg between floor and tub) is never a good idea - it is a matter of when it will crack, not if. The cracks between the tile in your photo, however, indicate an unstable subsurface, which may very well be from water constantly getting between the tile and tub over the years, and rotting the subfloor.

If you're pulling it all up, first repair/level the subfloor, then if possible, lay down an uncoupling membrane (schluter, wedi, or redgard each have products i believe.) This will make sure that movement of your subfloor doesn't transfer to the tile field. Some are suitable for installation of under-floor heating elements as well.

Next, while a bit more expensive and slightly harder to work with, i suggest a polymer modified grout as they are less prone to shrinking, bond better, and are more stain resistant (epoxy grouts are another option, but may be even harder to work with.)

And finally, don't grout between the tub and tile - that's what caulk is for. And not the cheap acrylic latex stuff - you at least need silicone, which you can often find from the same company that made your grout, color matched. An even more durable option would be an MS-polymer sealant/caulk, but you'll be pretty limited in color choices with that option.

Grout is going to need to be redone over time. Especially in those areas that take on any flex at all (like right next to the tub).

Rust would indicate metal of some kind. Either a trim piece or the tub insert itself is like cast iron (which in that case you would be able to keep it).

Looking at the photos, that seems to be very normal and expected grout wear. You can dig the grout out and re grout, and then caulk along that tub line.

Or yeah if you want to just redo the whole floor. Just get a sledge hammer and start breaking it up.

But none of that is alarming at all.

Looks like lots of good advice here already.

Idk how handy you are, but tile/grout was not nearly as hard to do as I expected. It just takes a long time to DIY this type of project. But it is immensely satisfying to answer "I did that" when someone compliments your finished work.

The only tool you might need that is not something most homeowners have is a wet saw. I bought a cheap one for almost the same price as renting a nice one for a weekend. And it has helped me do two bathrooms and soon my kitchen.