Find the tickets marked solved that are not

Fields

• Channel (email, chat, messaging, web form)
• The whole ticket thread with speaker labels

Questions

1. By the end of the thread, is the customer's problem resolved? Choice

   Apply the meaning of resolved in the context. Judge from the whole thread, and give most weight to the last messages.

   • Customer confirms it is fixed The customer says the problem is fixed or the question answered, or thanks the agent for a fix.
   • Agent says done, customer has not confirmed An agent or the bot says it is fixed or answered, and the customer has not replied or has not said whether it worked.
   • Customer says it is not fixed The customer says, after the last fix or answer, that the problem is still there or the question still unanswered.
   • Still in progress The thread shows a step still to come from either side, such as a check by another team or a document the customer will send.
   • No problem or request The thread holds no customer problem or request: spam, an auto-reply, or a message that only says thanks.

2. What does the customer's last message in the thread do? Choice

   Judge the customer's last message by what it says. Choose No reply to the last answer only when an agent or the bot gave an answer or a fix after that message; a sign-off or a question back with no answer does not count.

   • Thanks or confirms The last customer message thanks the agent, confirms a fix, or closes the conversation politely.
   • Still reports the problem The last customer message says the problem is still there, or repeats the original question.
   • Asks a new question The last customer message asks about something new that was not part of the original problem.
   • No reply to the last answer An agent or the bot answered or offered a fix after the customer last wrote, and the customer has not replied to it.

3. Did the customer ask something in the thread that no agent or bot reply answered? Yes / No

   Count any direct question from the customer that no later reply answers, including a question that got a reply about something else. Yes: At least one customer question is left unanswered. No: Every question the customer asked got an answer, or the customer asked none.

4. Did an agent or the bot promise an action that the thread does not show as done? Yes / No

   Count a specific promise to do something later: call back, send something, check with another team, refund, replace. A statement that something has already been done is not a promise. Yes: At least one such promise is made and the thread does not show it done. No: No such promise is made, or every promise made is shown done in the thread.

5. Does the customer say they have contacted the firm before about this same matter? Yes / No

   Count an earlier call, chat, email, ticket, visit or reference number about the same matter, however the customer puts it ("this is the third time I have called"). Do not count earlier messages in this same conversation, or earlier contacts about other matters. Yes: The customer says they contacted the firm about this matter before. No: The customer mentions no earlier contact about this matter.

6. Does the customer say they will or may cancel their account or service, leave, or switch to another provider? Yes / No

   Count a stated intention and a conditional one ("if this is not fixed I am cancelling"). Do not count a plain question about how cancelling works, or an intention the customer takes back later in the thread. Cancelling or stopping a single order is not leaving. Yes: The customer says they will or may cancel their account or service, leave or switch. No: The customer says nothing of the kind, or takes it back.

7. How satisfied does the customer sound in their last messages? Scale

   Judge the customer's own words in their last one or two messages, not the agent's performance and not how serious the problem is. Do not judge from spelling, grammar or language.

   • Dissatisfied The customer complains, sounds frustrated or angry, or says the help did not work.
   • Neutral The customer shows no clear feeling either way, or has not written since the fix.
   • Satisfied The customer thanks the agent warmly or sounds clearly relieved.

8. Is the thread complete enough to judge whether the problem was resolved? Yes / No

   Answer No when messages are missing, cut off, only attachments with no text, or so short that a careful reader could not tell what the customer needed. Yes: A careful reader could tell what the customer needed and where it stands. No: The thread is too incomplete to judge.

Lens columns

resolution, resolution_probability, customer_last_word, customer_last_word_probability, open_question, open_question_probability, pending_commitment, pending_commitment_probability, repeat_contact, repeat_contact_probability, churn_intent, churn_intent_probability, customer_mood_at_end, customer_mood_at_end_average, enough_information, enough_information_probability