Before you use an LLM to flag risky conversations
Language models can read every call and chat a bank handles. Whether that helps depends on decisions made before the first prompt is written.
Amogh Suman
Banks and insurers record a very large number of conversations: sales calls, service chats, complaint emails, agent notes. Only a small sample is ever reviewed by a person. Large language models make it possible to read all of them, looking for complaints that were never logged, mis-selling, abusive behaviour or signs of fraud.
That is a real opportunity. It also goes wrong in predictable ways. Here are the questions I would ask before starting.
What exactly are you trying to catch?
"Find risky conversations" is not a specification. A model needs a defined list of flags, each with a definition and examples. "The customer was told a product had guaranteed returns" is a flag. "Something felt off" is not.
Writing the flags down is most of the work, and it is work for compliance and business teams, not for engineers. If the people who own the risk cannot agree on what a flag means, no model will apply it consistently.
Who checks the model?
Every flag a model raises will be one of two things: a real finding or a false alarm. Every conversation it lets through may hide a missed finding. You need to know the rate of each, and the only way to know is to build a labelled set of conversations that people have reviewed carefully, and test against it.
A model that flags everything is useless. A model that flags nothing looks perfect until the regulator finds what it missed.
Plan for human review from the start. In a regulated setting, a flag is the start of a process, and a person has to decide what happens next.
Which language is the conversation in?
Indian customer conversations mix languages within a sentence, use Roman script for Hindi, and include local terms for financial products. A model that performs well on clean English text may do much worse on real transcripts. Test on your own data, in the languages your customers actually use.
Where does the data go?
Customer conversations contain personal data, and sometimes financial and health information. Sending them to a model hosted outside the institution raises questions under the DPDP Act, under sector rules on outsourcing and data localisation, and under the institution's own policies. Some conversations should not leave the building at all.
When can the human step back?
My view is that a human should stay in the loop for any flag that leads to action against a customer or an employee. Machines can sort, prioritise and summarise. Decisions about people need a person who can be held accountable.
The good news is that the first value often comes before any action is automated. Reading every conversation, rather than a small sample, shows patterns nobody could see before. Start there, measure honestly, and widen the model's role only as the evidence supports it.