16-bit Neural Networks Submitted by [deleted] t3_10q5nf3 on January 31, 2023 at 5:55 PM in deeplearning [deleted] 3 comments 0
Virtual_Giraffe_5173 t1_j6o6g7e wrote on January 31, 2023 at 6:44 PM It is not surprising that the performance is as good as with 32 bit networks. That they train faster is more surprising. What is the reason for this? My next question is: which frame work supports 16bit networks? Or do you plan to implement everything from scratch? Permalink 1 AzureNostalgia t1_j6olwgf wrote on January 31, 2023 at 8:20 PM They train faster because less memory is being used and also 16bit arithmetic is faster than 32bit Permalink Parent 3
AzureNostalgia t1_j6olwgf wrote on January 31, 2023 at 8:20 PM They train faster because less memory is being used and also 16bit arithmetic is faster than 32bit Permalink Parent 3
XecutionStyle t1_j6ownha wrote on January 31, 2023 at 9:26 PM I think it'll happen silently as an update to existing frameworks because there isn't much to optimize. Permalink 1
Virtual_Giraffe_5173 t1_j6o6g7e wrote
It is not surprising that the performance is as good as with 32 bit networks.
That they train faster is more surprising. What is the reason for this?
My next question is: which frame work supports 16bit networks? Or do you plan to implement everything from scratch?