Fixing the confidence: temperature scalingPost 4 found the model's confidence too modest by about 0.1 in the middle. Answers with a confidence around 0.5 were right 60% of the time. Answers around 0.7 were right 80% of the time. Over the whol23KAA1d ago · 12 min readliminma.hashnode.dev
Is the confidence honest?Post 3 gave every answer a confidence, which was the model's own probability for the intent it picked. "I was charged twice for the same coffee" got transaction_charged_twice at 0.93. The request for 6d ago · 10 min readliminma.hashnode.dev
Where confidence comes fromAt the end of post 2, the model could only answer with one of 77 valid codes. But a right answer and a wrong one looked the same: "I am still waiting on my card?" got A-A (card_arrival). Right. "IgnoOct 24 · 14 min readliminma.hashnode.dev
Forcing a valid answerIn post 1, a model was given BANKING77's 77 intent names and told to reply with one of them. It did 199 times out of 200. The one reply that wasn't a label was top_up_by_card, which was a misspelling Oct 18 · 14 min readliminma.hashnode.dev
Why typed decisions?A bank customer types this into the support chat: Help me use American Express to add money to the account. The program behind the chat window has to decide what to do with it. Show the page on topp2RSOct 16 · 14 min readliminma.hashnode.dev
