First, the average thickness of the cortex is more like 2.5 mm

Second, we aren't explicitly measuring ig each voxel is cortex or non-cortex with a binary 1-0 mask. The way that cortical thickness is measured is by looking at the signal intensity to each voxel, and an algorithm estimates the barrier between gray matter and white matter, or gray matter and the dura matter on the outside of the brain. It does so by considering the intensity of this voxel relative to deeper voxels that are clearly white matter, or voxels that appear more likely to be gray matter.

So let's pretend a voxel that a cortical voxel would have an intensity value of one. Let's say that a voxel of white matter has an intensity value of 0.5

If you encountered a voxel that was roughly 0.75, you could infer that roughly half of that voxel was in the cortex.

Free surfer runs the algorithm across the entire image and attempts to estimate the inner and outer barrier using this approach. I simplified it of course, it's not really saying 0.75 and half the voxel, the modeling is more complicated than that and frankly I don't understand it more than that.

But, the long story short is that by looking at signal intensity the different voxels we can build an estimate a model of where the ribbon of the cortex is relative to the edges of that voxel.

Interestingly these estimates are very reliable across MRI scans. If I scan you today and scan you next week, most cortical thickness measurements will be very stable

Then once you have the outer ribbon, representing the outside of the cortex, and the inner ribbon representing the inside, you can place a perpendicular vertex in that section and estimate the thickness of the cortex

The process isn't perfect, and errors are not uncommon. Particularly in lower quality MRI scans.

I hope that help explains it