Inspect AI in Production: A Hard-Nosed Review of Agent Evals, Logs, and Model-Upgrade Gates

We installed Inspect AI and attempted an offline scorer contract check, then developed a regression-task walkthrough around its task, solver, scorer, and agent interfaces. The framework is compelling, but we have not validated log reproducibility, grading stability, or model-upgrade cost accounting.