How to Correct an LLM Judge's Pass Rate With 100 Human Labels

The pass rate a grading model reports is part your product and part the grader's own mistakes. Measure those mistakes on a small set a person has checked, and you can subtract them back out.