Math is hard for people too, and I don't think GPT 4 is worse than the average person when it comes to math. In many cases, math requires abstract multiple step processing which is something LLM's typically aren't trained on. If these models were trained on processes rather than just content, they'd likely be able to go through the steps required to perform mathematical operations. Even without specific training, LLM's are starting to pickup the ability to perform multiple step calculations, but we're obviously not all the way there yet.