Built a tiny tool after my own validation numbers kept lying to me (metrictrust)

Hey everyone, while fine-tuning small models, I kept running into the same trap three times over: the model hit its best score mid-training, and I ended up keeping a worse final checkpoint train accuracy was much higher than eval — memorizing, not learning — so the number I was looking at overstated real performance the run oscillated, so the final number was really just one noisy sample, not a measurement Loss/metric loggers show you the curve, but none of them tell you when the number at th...