Dropping Candidates Due to Fixable Defects — A Story of Measuring Factory Flaws as Product Personality

In the selection of voice model candidates, we included 'resistance to generation breakdown' as an evaluation metric, but that breakdown was only a manufacturing defect that did not carry over to the post-trained model. This is a discussion about whether the metric is measuring the nature of the product or the state of the process.