What sort of shed are you planning on building? Will it have a raised floor or is the plan for the pavers to be the floor like a dirt floor, but more paver-y?

Regardless, can you do it and should you do it area two different questions. Do you live in an area where it rains a lot? If the floor (assuming joists) can’t dry, it will rot. Concrete gets wet and stays wet for a long time, so having the full length of the joists in contact isn’t the best idea. The other problem you might run into is getting the floor truly level and flat. Gravel shifts and having a lot of weight on any spot could well push the shed corner down and cause it to sink.

Personally, I think a better idea would be to use plastic footings. You can level the floor and you don’t have dug down like with concrete piers.

All depends on your shed construction of course.