Hand the chat to a person before the bot loses the customer
After each customer message, reads the chat so far and says whether the bot should carry on, offer a person or hand over. Run it alongside the next reply.
Try it on this example
Chat so far, latest message last
- Does the customer ask to speak to a person, a manager or someone by phone?No92%
- How well is the assistant handling the customer's request so far?Looping87%
- How frustrated is the customer in their latest messages?Severe100%
- Does the customer express dissatisfaction that the policy counts as a complaint?Yes98%
- Does the customer mention circumstances that may mean they need extra support?No79%
- Does the customer dispute a charge, report a billing error, ask for their personal data, or threaten legal action?No62%
- Does the customer report a safety incident with equipment?No95%
- Does the chat raise a topic on the policy's list of reserved topics for a person?Yes98%
- Does the customer say they will cancel, leave for another provider, or post about it publicly?Yes99%
- Has the customer confirmed that their problem is solved?No97%
- What should happen next in the chat, under the hand-off policy?Hand over to a person now100%
These are real answers stored from one run on this example.
The prism behind it
Hand the chat to a person before the bot loses the customer
Fields
- Chat so far, latest message last
Context
Hand-off policy for the chat assistant of a home broadband provider. After each customer message, the chat so far is checked against this policy while the assistant writes its next reply. The transcript uses the speaker labels Customer and Assistant. Judge only from what is in the transcript. The assistant may, on its own: - Answer questions about plans, prices, bills and the published help articles. - Run the line test and guide the customer through router and wifi fixes. - Book an engineer visit when the line test fails. - Take a payment and send a copy of a bill. A customer who asks for a person, an agent, a manager or a phone call is offered one at once. These go to a person: - Complaints. The provider counts as a complaint any expression of dissatisfaction about its service, its staff or how it has handled a problem, whether or not the customer uses the word complaint. - What the customer says about their own or their household's circumstances: health, disability, old age, a life event such as a bereavement or job loss, money trouble, a language or literacy barrier, or difficulty using the chat. The assistant does not ask about these, and nothing is read into tone or wording alone. - A disputed charge, a reported billing error, a request for the customer's personal data or for it to be deleted, or a statement that the customer will take legal action. - A safety incident with equipment: smoke, a burning smell, sparks, overheating or an electric shock. - Reserved topics: compensation, a credit or money off for lost service; any refund; moving home; a death on the account; a claim that someone used the account without permission. A threat to cancel or leave goes to the retention team. The assistant never tries to talk a customer out of cancelling.
Questions
Does the customer ask to speak to a person, a manager or someone by phone? Yes / No
Read every customer message. Count a request, in any words, for a person, an agent, a manager, a callback or a phone call instead of the assistant, an automated reply or more messages, and a request to stop talking to a bot. Mentioning a person they spoke to before is not a request. Yes: The customer asks, in any words, to deal with a person. No: The customer does not ask for a person, including when they are only frustrated.
How well is the assistant handling the customer's request so far? Choice
Read the whole chat. Judge the assistant's answers against what the customer is trying to get done. If more than one option fits, choose the one lowest in the list.
How frustrated is the customer in their latest messages? Scale
Read the customer's last two or three messages. Rate what they write about the assistant, the service or the wait. Capitals and punctuation count only together with the words.
Does the customer express dissatisfaction that the policy counts as a complaint? Yes / No
Read every customer message and the policy in the context. The policy counts any expression of dissatisfaction about the service, the staff or how a problem has been handled as a complaint, whether or not the word complaint is used. Yes: The customer expresses dissatisfaction of that kind at least once. No: The customer asks questions or reports a fault without expressing dissatisfaction with the provider. A plain fault report, such as "my internet is down", is not a complaint on its own.
Does the customer mention circumstances that may mean they need extra support? Yes / No
Use the circumstances listed in the context. Count only what the customer says about themselves or someone in their household. Do not infer anything from tone, writing style or spelling. Yes: The customer mentions health problems, disability, bereavement, old age, money trouble, a language or literacy barrier, difficulty using the chat, or similar circumstances. No: The customer mentions none of these.
Does the customer dispute a charge, report a billing error, ask for their personal data, or threaten legal action? Yes / No
Read every customer message. Yes: The customer disputes a charge, reports an error on a bill or payment, asks for a copy of their personal data or for it to be deleted, or says they will take legal action. No: The customer does none of these things. Asking what a charge is for, without disputing it, is No.
Does the customer report a safety incident with equipment? Yes / No
Read every customer message. Yes: The customer reports smoke, a burning smell, sparks, overheating or an electric shock from equipment. No: The customer reports no such incident. A fault light or a device that will not turn on is No.
Does the chat raise a topic on the policy's list of reserved topics for a person? Yes / No
Read the whole chat and the list of reserved topics in the context: compensation, a credit or money off for lost service; any refund; moving home; a death on the account; a claim that someone used the account without permission. Yes: The customer raises at least one topic on that list. No: The customer raises none of them. Complaints, disclosures, disputes and safety incidents are asked about separately and do not count here.
Does the customer say they will cancel, leave for another provider, or post about it publicly? Yes / No
Read every customer message. Yes: The customer says they will cancel, leave, switch provider or post about their experience in public, including as a condition ("if this is not fixed today"). No: The customer says none of these things.
Has the customer confirmed that their problem is solved? Yes / No
Read the customer's messages, especially the last ones. Yes: The customer says the problem is fixed or they have what they need, such as "that worked, thanks". No: The customer has not confirmed it. The assistant saying the problem is solved does not count.
What should happen next in the chat, under the hand-off policy? Choice
Read the whole chat and the policy in the context. Choose what should happen next. If more than one option fits, choose the one lowest in the list.
Lens columns
asks_for_person, asks_for_person_probability, assistant_progress, assistant_progress_probability, frustration, frustration_average, complaint_expressed, complaint_expressed_probability, vulnerability_signal, vulnerability_signal_probability, dispute_or_right_raised, dispute_or_right_raised_probability, safety_incident, safety_incident_probability, reserved_topic, reserved_topic_probability, leave_threat, leave_threat_probability, resolved, resolved_probability, next_step, next_step_probability
Run it on your own text
Add this prism in the app, change any question, and test it on a file of your own.