You do you! There is a reason that millions of librarians over hundreds of years have gone for vertical, but if you keep them clamped tight enough and somehow support them all every single time you want to take a book out, yes, I suppose you can store them spine up. Good luck with that!