Kalpak Kale

Thanks Kartik. Tone changes were the single biggest source of regressions for us, which is why rule 2 (a regression blocks until explained) exists at all. On your question: a document-issue tag by itself does not become a case, because the suite tests the bot, not the corpus. Those go to the content owner as a governance item. It becomes a case when the tag exposes a behaviour gap: for example the bot answered confidently from outdated material when the correct output was “the document does not say”. That failure is the bot’s, so it gets a case and a weight. Roughly, document issues feed the content backlog, and the bot’s reaction to bad documents feeds the suite.

Good catch, and you are right. Keying on the normalised current question alone can replay an answer from an earlier context when the short query depended on the previous message. The fix is exactly what you describe: key on the resolved standalone query (after the expansion step) plus the corpus version, so a changed document invalidates the entry as well. Your paired case is a better test than the one we had. Same short question after two different product references in one session, answer must follow the new one. That is going in as case 22. Thanks for the improvement.