Use two layers, first is base rock tamped down tight and properly sloped then a couple of inches of masonry or paver sand. Finally paver sand is used to fill the cracks after the bricks are laid. Do not use pea gravel. The roundness of pea gravel does not allow it to tamp down tight.

You also need a border to keep everything tight.