Actually fats are not much more complex than sugar molecules, if at all. Fats are essentially just 3 straight chain hydrocarbons that may or may not be saturated. Once lipolysis begins it is actually a faster process than breaking down glycogen and sugar. It’s just the body will delay initiating the process because our body prioritizes the use of glucose over fats for energy for various reasons.

Google a picture of triglycerides and a picture of glycogen and see which is more complex.