+40% CUDA +160% MPS Throughput Improvement on PyTorch Adam Optimizer

Hi PyTorch community, I recently published a modified PyTorch Adam optimizer implementation that achieves up to a +40% CUDA and +160% Mac MPS speedup in step execution and training throughput. Free size-limited fully working benchmark example and code at GitHub - yangofzeal/adam: 40% faster replacement for torch Adam Optimizer on cpu and GPU · GitHub