My demo refused 96% of off-topic questions but only 39% of tricky ones. I know because of a test set

A golden set of 100 labelled examples is the cheapest acceptance test for any AI feature. How to build one, and what it exposed in my own document Q&A demo.