the.bay.news

Comparing INT4 and NVFP4 Palettes on Real Gradient Tensors

DEV Community
Comparing INT4 and NVFP4 Palettes on Real Gradient Tensors
A palette study on synthetic blocks said evenly-spaced INT4 beats NVIDIA's FP4 grid once a Hadamard rotation gaussianizes the data, and loses by 2.3x without it. On real gradient tensors it wins either way, 41 of 45 without the rotation, because my stand-in for unrotated data put the outliers in the wrong place.

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.