I've lived in both places after living in Baltimore. They are both predominantly white. West Chester felt somewhat more diverse, but it's also about 3-4 times the size and, as you mentioned, has the public university, which is a large contributor. I would look at the demographics of the school systems to better gage what other the young families you'll be around/your kids peers, but I imagine they would be very similar.

Media is much much more convenient for getting into Center City, and depending on where you live, you can easily walk to the train.