the.bay.news

[ROCm][MI200] linear_cross_entropy recomputed logits use backward-only FP16 alternate GEMM path

PyTorch Forums
[ROCm][MI200] linear_cross_entropy recomputed logits use backward-only FP16 alternate GEMM path
Description I investigated three fp16 linear_cross_entropy test failures on gfx90a / MI200 originally reported in ROCm/TheRock#6993 and was able to reproduce them and trace the numerical behavior further. The failures affect the reduction='none' backward path for the balanced, compact, and auto accumulation policies. Related PyTorch CI tracking: pytorch/pytorch#191552. Root cause analysis The issue does not appear to be caused by a rocBLAS or hipBLASLt kernel defect. In the reduction='none' ...

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.