Because most people who had something better to do would skip and you'd end up being judged by a jury of people who had nothing better to do.

Small edit: as a non-American I sometimes envy your jury system. Here the outcome of a court case strongly depends on whether you get a happy judge or an 80-year old communist aparatchik who hates your tattoos or your choice to be gay.