Banking - Scaling up customer service with chatbot

Banking - Scaling up customer service with chatbot
Uses cases
Voice of customer



  • Provide rigorous training to build a trustworthy ML model

  • Control quality of large training datasets

  • Scale to address multiple languages


  • Human in the loop: advanced quality control features including honeypot and consensus

  • Intuitive collaboration workflows between data scientists and labelers making learning and iteration quicker


A large European banking client found itself having limited capability when handling customer service functions manually by answering phone calls. Customers need to wait in lines and wait for long periods which lowers consumer satisfaction and loyalty.

When the bank decided to implement artificial intelligence to build chatbots at scale, Kili Technology became its trusted partner to facilitate training datasets to enrich the quality of the chatbot and monitoring in production.

How it all started

The need for chatbots to accompany customers

Like most banking institutions at the time, a large European bank client found itself needing to handle customer service to answer daily questions or complaints from customers through phone calls by customer service operators. The old usual process was typically illustrated by a customer dialing into a customer service number and having to wait in line until the operator picks up – and having to listen to a random “waiting song” repeated several times over. As the volume of customers dialing in is always far greater than the operators handling the calls, to customers this “waiting song” is playing in the loop without end.

Their objectives :

When these customers have urgent queries, this unreachability became a pivotal point against the bank, as they see their customers thinking to turn heads to alternatives. To solve this issue, an AI chatbot development at scale was considered a highly attractive alternative to the old-fashioned customer service calls.

We conducted a survey of our banking customers, and major feedback was given to how long they had to wait just to ask questions. There was a barrier – we were unreachable to them.

Head of Customer Success

This AI chatbot would erase the high barrier between the bank and customers easily, responding to each one of the customer’s questions and messages in real-time, at all times. Additionally, the chatbot also allows self-care development, in which customers can get their answers quickly themselves and only escalate the conversation to be delegated to human operators only when it is added value.


The challenges of building an AI chatbot at scale

Building an artificial intelligence model for email processing is not without its challenges. First of all, it is challenging to create a classification scheme that is simple enough to be understood by the AI model. Second, there should be a strategy that allows the bank to locate risk on mistakes in classification.

Moreover, another challenge is to facilitate the AI automation model to have room for data correction and reclassification of data when there is a mistake in identifying information or attached documents within the email. Human feedback can enrich the model and make it more robust. To address these challenges, the bank then needed to make sure that the AI model is trained rigorously with datasets that are accurately labeled to ensure the high-quality output of performance.

…the point is to be able to investigate where the AI bot got lost and tell whether this part of the conversation should have been understood well. If we find gaps between intent, we shall add this missing intent so next time the bot will perfectly understand.

Head of AI Labs

To address these challenges, the bank then needed to make sure that the AI chatbot is trained rigorously with bulk amount datasets that are accurately labeled entities covering the variation of intent in multiple languages.

Issues with raw data from banking documentation

  • Complex layouts

  • Nested tables

  • Handwriting

  • Multiple docs in a single PDF

  • Symbols

  • Images, logos, stamps

  • Noisy images

  • Contextual relationships (footnotes)


A versatile training data platform to implement Intelligent Document Processing (IDP)

As the bank is already aware, the data annotation process is vital to produce high-quality datasets to train the chatbot to understand the variety of intent. Based on the challenges mapped, the bank was looking for a data annotation company that could provide a simple tool with rigorous quality management and flexibility in collaboration. After testing several tools offered in the market, the bank considered Kili Technology to be superior to its competitors.

Working with Kili, the bank found it useful to leverage advanced features such as consensus and honeypot to control the data quality to make sure utterances are interpreted well with the correct intent. On top of that, the Kili platform also simplifies the collaboration between data scientists and annotators during the labeling process, making learning and iteration quicker.


“It’s definitely worth it.”

Using chatbots, the bank sees huge cost savings. The cost with customers, when conducted with a human operator, costs 3 euros/interaction. The bank was conducting 1 million interactions as a basis to produce another 4 million incremental interactions. With human operators, this would cost EUR 3 million. Implementing the chatbot has reduced the need for human interaction by 50%, therefore saving costs by EUR 1.5 million.

The impact is massive. We receive customer compliments praising our better customer service. We got around 60% ROI, which is pretty well. It’s definitely worth it.

Head of Customer Success

Lesson Learned

  • Labeled data plays a vital role in determining the accuracy of an AI chatbot, hence investing time and effort in data labeling is of the utmost importance to ensure the success of projects.

  • Human in the loop during the chatbot training and production is absolutely critical to achieving the great capability of the chatbot to create impact: save cost and improve productivity.

  • It is important to select a suitable data annotation partner for your chatbot development. Factors such as robustness, meticulous quality management, and simplicity of collaboration are key.

Related Resources

AI For Compliance: What, Why and How?

AI For Compliance: What, Why and How?
Kili’s guide to the new AI paradigm

Kili’s guide to the new AI paradigm
Opinion Classification with Kili & HuggingFac...

Opinion Classification with Kili & HuggingFac..

Get Started

Get started! Build better data now.