Measure the Judge Before You Trust It: Self-Consistency Comes Before Human Agreement

Here's a question almost no eval pipeline can answer: if you asked your LLM judge to score the exact...