Likely just superficial drywall crumbling because of moisture. Agree that most common source is the flange. Seems scary but not the hardest job in the world. You can use a screwdriver to test the wood. If you can stab into the wood and it feels more like stabbing into dense bread/cake it’s rotten. Also, if you’re walking around upstairs and it feels bouncy or the floor is squishy, it’s rotten.

If it’s solid, leave it. Otherwise, replace rotten pieces. Subfloor is pretty easy to replace. Talk to a pro if the joists are damaged. Leave water turned off and dry the exposed 2nd floor joists and subfloor with a fan in the kitchen for 24ish hours. Next, once everything is dry, put a couple of tablespoons of bleach into a 32oz spray bottle and fill the rest with water. Mix it up well and spray any areas just enough to coat that we’re exposed to the moisture. This should kill any mold spores that may have set up. If there’s black or slimy stuff in there, throw a good mask on and clean it up before drying. Other commenters had good instructions for re-seating the toilet on a fresh flange. Patch in drywall once all is done. Also not a bad idea to just throw a small frame around the hole with some 1x4 wood if you’re not too worried about aesthetics and leave the center piece in the middle detached for future access. Plan on a weekend of work.