There's some proper looney tunes ideas in this thread. Hard to work out why people are so unhinged over such a relatively simple idea, but horses can be lots of different sizes (they still are today!) and still be used to carry riders. Do a simple google search and you can find horse sizes can vary quite a lot. People ride ponies all the time, quite happily.

Absolutely no idea why people are talking about breaking horses backs and stirrups and ancient paintings. Lay off the youtube videos for a little while.