Skilled forensic anthropologists usually can, but the issue is that most of the graves we're talking about here have never been studied this way, and never will be, because the bones do not exist anymore. In some, the bones disintegrated naturally before the grave was even excavated. There were no surviving bones found in the Sutton Hoo grave, for example. For many though, the bones were simply discarded because the artifacts were all that people paid attention to. This was pretty common practice up until pretty recently, and definitely was during the Victorian antiquarian boom. They dug up the grave, found a sword, wrote the skeleton down as male and Germanic, and then threw the bones in the trash. We are still operating off of that data today.

But it wouldn't even help you to have the bones in this case. A skeleton can't tell you what language someone spoke, or what terms they would use to describe their own background. Not even their DNA can tell you that. Identity is not biological.

As an aside, just because someone had a skeleton we might describe as biologically male does not mean that that person or their society saw them as a man.