If the hardware is optimized for it, there probably is not a huge difference in speed, but the performance gain is probably negligible too.

The real reason people dont use 64bit is mainly memory usage. When you train a large model, you can fit much bigger 32bit/16bit batches into memory and thereby speed up training.