About UK Power Networks
UK Power Networks is the largest electricity distributor in the UK. It maintains electricity cables and lines in London, the East, and the Southeast of England. UK Power Networks delivers energy to 19 million people across its remit. It ensures that electrical networks are safe, secure, and reliable. It is a key player in helping the UK meet Net Zero and focuses on supporting renewable energy, low-carbon heating, and electric car chargers.
UK Power Networks drives innovation in the energy sector by constantly engaging in technological advancements. The company leverages its data assets and utilises machine learning to tackle Net Zero, automate internal processes and provide reliable, cost-effective solutions to customers. Customer satisfaction lies at the heart of their business. The company cultivates the customer service culture and ensures that all customers, especially the vulnerable ones, are protected.
Overview of the challenge
Customers are increasingly using digital means to get in touch, and one of the challenges faced by the customer service team at UK Power Networks is a huge number of emails coming each day (300-400 emails with a 10% increase year on year). Those emails can be classified into three categories. The first category includes requests for actions (jobs), such as preparing quotations or disconnecting clients. The second category includes questions on existing jobs or other inquiries about the business. The final category consists of emails of lower priority to the energy distributor, such as automatic replies or meeting notifications. The customer service team used to manually review incoming emails and assign categories to them, indicating their urgency. This process was time-consuming and prone to error, which could lead to UK Power Networks’ delays for some customers and in the worst case service level agreements (SLAs) not being met.
UK Power Networks partnered with Databricks, Microsoft and CKDelta to create an automatic solution that would speed up this process. The effect of that collaboration was creating a Customer Virtual Agent. It’s a new Outlook inbox experience, enhanced by intelligent features such as email classification and summarisation, using Large Language Models (LLMs) on Azure Databricks. The solution was developed as a Proof of Concept to test the capabilities of generative AI for a customer service use case.
Developing the solution with CKDelta
CKDelta has been UK Power Networks’ technical partner across several innovation projects using machine learning, including Spotlight, Envision and Optimise Prime. CKDelta builds data-driven AI applications and machine learning models, empowering customers to achieve sustainable, safe and efficient business outcomes.
As part of the CK Hutchison Holdings Group, CKDelta has access to uniquely enriched and continuously refreshed data from industrial-scale sources. The AI applications underpinned by this multiple-sector data present a rare opportunity to learn from the past and predict the future with confidence.
The Customer Virtual Agent was built in partnership with Databricks and Microsoft. Services provided by those partners are at the core of the developed solution. Under the hood, the email processing workflow runs on the Azure Logic Apps platform. Each incoming email triggers this workflow. In the first step, it saves email bodies, subjects, and attachments in Azure Blob Storage. Then it moves to the computational part of the solution. That part takes place in the Databricks platform.
Databricks allows using PySpark for data preprocessing easily and efficiently. PySpark was an essential tool to transform UK Power Networks’s email dataset. It consists of large volumes of text data which is easy to load into PySpark DataFrames. They can then be viewed, explored, and interacted within Databricks’ notebooks. The email data was cleaned with the help of LLMs. The UK Power Networks’ instance of the GPT-3.5 Turbo model was accessed directly from Databricks using Azure OpenAI API. The model was used to identify parts of the emails important from the point of view of the customer service team. The instructions, containing UK Power Networks’ inside knowledge, are a part of the prompt passed to the model. LLMs are also being used to assign email categories and provide summaries of the messages.
As mentioned in the previous section, three GPT-3.5 Turbo models are responsible for handling incoming emails. Each of the models has exactly one task to perform, i.e., identify important parts of the emails, assign categories, or summarise the content. Prompts passed to the models include domain knowledge that the customer service team at UK Power Networks uses to make decisions about incoming requests. Changing those prompts is easy and allows for quick improvements in the accuracy of the model. It also allows tailoring the models towards important business metrics. MLFlow experiments were used to test what prompts have the best impact on the model’s performance. Experiment tracking in Databricks is highly effective and with new LLMOps features introduced in MLFlow 2.4. it’s easy to compare prompts and track LLM-specific metrics. Databricks also allows to easily register LLMs and to access them for new incoming emails.
In the last step, the predicted email categories and summaries are sent back to Outlook. A category is added to the original email as an email tag. This way members of the customer service team can see the focus of the email, without even having to review its content.
An email summary is then added in a different colour at the beginning of the original email. This makes the division between the original content and the LLM output clear and intuitive. Most importantly, the solution enhances the experience of the customer service team, without changing the way they proceed. It doesn’t require any additional work on their behalf nor modifies the approach that has proven to be optimal for them.
Benefits of the solution
The Customer Virtual Agent saves time that the customer service team can use to provide more customised support to their clients. Having out-of-the-box categories for all emails speeds up reviewing emails and minimises the risk of missing SLAs and receiving penalties. Summaries are particularly useful in the case of long chains of emails between customers and UK Power Networks departments. Reading a summary can give subject matter experts (SMEs) an idea about the nature and urgency of the request, as well as a list of actions already taken by UK Power Networks. It can also be over 10 times faster than reading a very long chain of emails. The outputs provided by LLMs improve business processes in UK Power Networks and directly impact customer satisfaction.
Next steps for the Customer Virtual Agent
The solution has proven that LLMs can be successfully used to optimise business processes connected to customer service. The SMEs found the solution effective and time-saving, highlighting the reliability of outputs provided by LLMs. It will have a significant impact on the focus of their work, which will no longer require manual classification of incoming emails. Considering the positive reception of the solution, the next step will involve productionising it to make Customer Virtual Agent a permanent part of the process that emails go through. Over time the solution could become more customised, addressing ongoing feedback from the customer service team. Thanks to Databricks Model Serving it’ll be easy to test different model families, including Llama and Mistral models, to compare their results with the initially picked GPT-3.5 Turbo. Model Serving provides a unified interface that enables managing multiple models with a single API, making future experimenting with different LLMs more agile. Those models could also be tested for other potential features of the solution, such as alerting about the urgency of requests or providing drafts of responses.
