Use cases

Score every call, chat and email on your QA scorecard

Answers each line of your QA scorecard on every conversation, so QA analysts review the ones that need it. Nothing here decides anything about an agent.

Ask for an invite

Try it on this example

Example · A duplicate direct debit blamed on the bank

Channel (call transcript, chat, email): chat

The whole conversation with speaker labels

[14:02] Customer: Hi, my water bill direct debit has gone out twice this month. Two lots of £42.60 on the 1st and the 2nd. [14:03] Agent (Dan): Hi, thanks for contacting Tarnside Water, my name is Dan. I'm happy to look into that for you. Can I take your postcode and date of birth first please? [14:03] Customer: EH6 6LT, 4 May 1971. [14:04] Agent (Dan): Thank you. And can I take your full name? [14:04] Customer: Glen Reid. [14:06] Agent (Dan): Thanks Glen, you're through security. I can see the two collections of £42.60 on the account, on 1 and 2 September. [14:07] Customer: Right, so can I have one of them back? It's left me overdrawn and I've been charged by my bank for it. [14:09] Agent (Dan): Looking at this, it looks like a bank error, so you'll need to speak to your bank about getting the second payment back. They can reverse it under the Direct Debit Guarantee. [14:10] Customer: Your company took it twice?? My statement shows TARNSIDE WATER DD against both payments, so it's on your side, not the bank's. [14:12] Agent (Dan): I understand. I'll raise it with our payments team so they can check what happened, and someone from that team will call you back within 2 working days. [14:12] Customer: Ok. So will I get the £42.60 back? [14:13] Agent (Dan): The payments team will confirm that when they call you. They will call the mobile number on the account. [14:13] Customer: Fine. That's the right number. [14:14] Agent (Dan): Great. Is there anything else I can help with today? [14:14] Customer: No that's all. [14:15] Agent (Dan): Thanks for contacting Tarnside Water, have a good afternoon. [14:15] System: Chat ended by agent.
  1. Did the agent confirm the customer's identity as scorecard line 1 requires?Done100%
  2. Did the agent own the issue, without blaming others or sending the customer elsewhere without help?No89%
  3. Did the agent tell the customer anything that contradicts the policies in the context?Yes98%
  4. Did the agent acknowledge the customer's situation when the scorecard calls for it?Missing when needed94%
  5. Did the customer have to repeat something they had already told the agent?No78%
  6. Did the agent tell the customer what happens next, who does it and when?Clear97%
  7. Did the agent end the conversation while the customer was still asking something or saying the problem was not dealt with?No79%
  8. Was the customer transferred, and did the policies call for it?Transfer the policy does not call for34%
  9. Did the agent give the customer money or a credit, and for what reason?No credit97%
  10. Did the agent ask for a good survey rating or tell the customer how to score the survey?No93%
  11. What did the agent promise to do after the conversation?Callback100%
  12. Is the conversation complete enough to judge it against the scorecard?Yes96%

These are real answers stored from one run on this example.

The prism behind it

Score every call, chat and email on your QA scorecard12 questions

Fields

  • Channel (call transcript, chat, email)
  • The whole conversation with speaker labels

Context

We are Tarnside Water, a UK water company. Our customer service is run by our own team and by an outsourced partner, and both use the same scorecard. The text is one whole conversation between a customer and a human agent. How the answers are used: - They choose which conversations a QA analyst reviews, replacing random picks. The analyst listens or reads, then decides. - Team leaders coach from rates over many conversations, and only after a person has reviewed the flagged ones. No single answer is a verdict on an agent, and nothing here is used for discipline, pay or ranking. - Judge what the agent did, never how they speak. Accent, fluency, grammar, spelling and word choice are not scored, as long as the customer's question is answered. - Lines that need system data, such as whether a refund amount was right, are checked by people or code, not here. Scorecard: 1. Identity: before discussing the account, confirm two of: full name, address or postcode, date of birth, account number. Not required when the customer asks only a general question. 2. Ownership: deal with the issue or take it to the right team yourself. Never blame another team, a supplier or the customer, and never send the customer elsewhere without help. 3. Policy: say nothing that contradicts the policies below. 4. Empathy: when the customer describes a problem that affects them, such as money, health, time lost or stress, acknowledge it in your own words. A routine request needs none. 5. Listening: the customer should never have to repeat something they already said. 6. Next step: before the end, tell the customer what happens next, who does it and when, whenever anything is left to do. 7. Closing: never end the conversation while the customer still has an open problem or question. 8. Transfers: transfer only when the policies say another team handles it. 9. Surveys: never ask for a good rating or tell the customer how to score the survey. Policies: - When we take a direct debit twice or for the wrong amount, it is our error. We refund the extra within one working day and never tell the customer to contact their bank. - Agents may give a goodwill credit for a service failure we caused, with the reason noted. Credit limits are checked by code. - Leaks, sewer flooding and supply interruptions are transferred to the operations desk. Billing, payments and meter questions stay with the agent. - Customers who say they are struggling to pay are offered our payment support options.

Questions

  1. Did the agent confirm the customer's identity as scorecard line 1 requires? Choice

    Use scorecard line 1 in the context.

    • Done The agent confirmed at least two of the listed details before discussing the account.
    • Not done when required The agent discussed the account without confirming two details, or confirmed them only afterwards.
    • Not required The customer asked only a general question and the account was never discussed.

  2. Did the agent own the issue, without blaming others or sending the customer elsewhere without help? Yes / No

    Use scorecard line 2. Blaming another team, a supplier, the bank or the customer counts against ownership, even if the agent later helps. Yes: The agent dealt with the issue or took it to the right team, and blamed no one. No: The agent blamed someone, or sent the customer elsewhere without helping.

  3. Did the agent tell the customer anything that contradicts the policies in the context? Yes / No

    Use scorecard line 3 and the policies in the context. Judge only what the agent said. Do not judge amounts or facts that need system data. Yes: At least one thing the agent said contradicts a policy. No: Nothing the agent said contradicts a policy, including when no policy topic came up.

  4. Did the agent acknowledge the customer's situation when the scorecard calls for it? Choice

    Use scorecard line 4. Politeness formulas alone ("thanks for waiting") are not acknowledgement.

    • Shown when needed The customer described a problem that affects them and the agent acknowledged it in their own words.
    • Missing when needed The customer described a problem that affects them and the agent did not acknowledge it.
    • Not needed A routine request where the customer described no such problem.

  5. Did the customer have to repeat something they had already told the agent? Yes / No

    Use scorecard line 5. Count a customer restating a detail or request because the agent missed it. Do not count a customer adding new information or confirming a detail when asked to for security. Yes: The customer repeated something the agent had missed. No: The customer did not have to repeat themselves.

  6. Did the agent tell the customer what happens next, who does it and when? Choice

    Use scorecard line 6. Judge from the last part of the conversation.

    • Clear Something is left to do, and the agent says what, who and when.
    • Next step missing Something is left to do, and the agent leaves out what happens, who does it or when.
    • Nothing left to do The issue was fully dealt with in the conversation.

  7. Did the agent end the conversation while the customer was still asking something or saying the problem was not dealt with? Yes / No

    Use scorecard line 7. Count closing a chat, ending a call or signing off an email while the customer is still asking something or says the problem is not dealt with. Yes: The agent ended it with the customer still waiting on an answer. No: The customer had no open question when it ended, or the customer ended it. A problem handed on with a next step, where the customer says they have nothing more, is No.

  8. Was the customer transferred, and did the policies call for it? Choice

    Use scorecard line 8 and the transfer policy in the context. Count only passing the customer to another of our teams or numbers. Telling the customer to contact an outside party, such as their bank, is not a transfer.

    • No transfer The agent kept the conversation and did not pass the customer on.
    • Transfer the policy calls for The customer was passed to the team the policies say handles the issue.
    • Transfer the policy does not call for The customer was passed to another of our teams, or told to call another of our numbers, for an issue the agent should have kept.

  9. Did the agent give the customer money or a credit, and for what reason? Choice

    Judge only whether a credit or refund was given and the reason the agent states. Do not judge the amount; code compares it with the limits.

    • No credit The agent gave or promised no refund, credit or payment.
    • For a failure we caused The agent gave a refund or credit and named a failure on our side as the reason.
    • Goodwill with no failure named The agent gave a credit to calm the customer or end the conversation, with no failure on our side named.

  10. Did the agent ask for a good survey rating or tell the customer how to score the survey? Yes / No

    Use scorecard line 9. Telling the customer a survey will follow, with no hint about the score, is allowed. Yes: The agent asked for a good rating or suggested a score. No: The agent did neither, including when no survey is mentioned.

  11. What did the agent promise to do after the conversation? Choice

    Pick the main promise the agent makes for after the conversation. Code checks later whether it was kept.

    • Nothing The agent promised nothing for after the conversation.
    • Callback The agent or another team will call the customer back.
    • Refund or credit Money will be refunded or credited later.
    • Visit Someone will visit the property.
    • Written follow-up A letter, email or text will be sent.
    • Escalation The issue will be passed to another team or a manager, with no callback promised.

  12. Is the conversation complete enough to judge it against the scorecard? Yes / No

    Answer No when the transcript or thread is cut off, has large gaps, or is so damaged by transcription errors that a careful reader could not follow what the agent did. Yes: A careful reader could follow the conversation from start to end. No: The text is too incomplete or garbled to judge.

Lens columns

identity_check, identity_check_probability, ownership, ownership_probability, policy_contradiction, policy_contradiction_probability, empathy, empathy_probability, customer_repeated, customer_repeated_probability, next_step_clear, next_step_clear_probability, premature_close, premature_close_probability, transfer, transfer_probability, credit_given, credit_given_probability, survey_steering, survey_steering_probability, promise_made, promise_made_probability, enough_information, enough_information_probability

Run it on your own text

Add this prism in the app, change any question, and test it on a file of your own.

Ask for an invite