Using ChatGPT for Task Classification

One of the biggest challenges we face at Spryngtime in AI-powered customer support is the wide range of intents behind customer support requests. From "Where's my order?" to "Does your product do X?", the needs of our users can vary tremendously, and the manner in which these requests should be handled differs just as much. As a result, the "one prompt fits all" approach simply doesn't cut it. Instead, we've had to become more creative in how we handle and classify the multitude of requests we receive daily. This blog post delves into how we've leveraged ChatGPT's power to achieve this.

The Problem: Classifying Support Requests

Typically, a support request like "Where's my order?" requires us to track down an order number and the associated shipping details. In contrast, a query such as "Does your product do X?" can be addressed by simply searching our documentation for the answer. These tasks are clearly quite distinct, and treating them as the same would lead to inefficiencies and bad responses. As our range of support questions increase, we can’t just keep adding more instructions onto a huge, “mono prompt”. We also realized that separating task-specific prompts (”agents”) performed better.

Ultimately, these different types of support questions required distinct prompts with distinct and different instructions.

Our challenge, then, was to effectively classify each request so that it could be addressed appropriately.

Step 1: An Agent-Driven Approach

The "output" of task classification problem would be inputted into what we refer to as "agents", essentially defined sets of actions. By having a prompt such as "Classify this task" for ChatGPT, we could identify the nature of a user's request and direct it to the appropriate agent. For instance, if the request was "Where's my order?", ChatGPT would classify the task as an order tracking request, and we could then handle this request through an agent designed for tracking orders.

The beauty of this system lies in its flexibility and scalability. With different agents set up for different task classifications, we can have highly focused and accurate prompts to handle the vast array of support requests, improving our response accuracy.

Step 2: Chain of Thought as a secret weapon

So, how exactly do we know which agent to use for a task? Task classification of course! Traditional task classification often involves creating large classifiers with meticulously labelled data. But with ChatGPT, our approach has been slightly different.

The first key component of our approach is the use of examples. By providing a few examples for each task category, we enable the model to better recognize and understand the nuances and variations in user requests. This allows for more accurate task classification and subsequently, more effective handling of user requests.

The second key element of our methodology is the incorporation of a 'chain of thought'. This concept, which comes from a research paper (read it here), revolves around guiding ChatGPT through a sequential, logical progression to reach a conclusion.

To make this clearer, consider the way we learn arithmetic. We understand that 1+1=2 because we start with one, then add one more, and end up with two. This stepwise logic is the essence of a 'chain of thought'. In the context of ChatGPT, we apply this concept by outlining a series of logical steps for the model to follow to arrive at a response.

If we take an example of classifying a request, we would initially instruct the model to figure out how to classify. For each classification, we would help the bot "reason" through why it might classify the task as such. For example, for a "Where's my order?" tasks, we would tell the bot "The customer is asking for updates about their order, therefore it should be classified as an order tracking request". After we classify, we would then guide the model to formulate a suitable response based on the order status. By implementing this systematic 'thought process', we have noticed a significant improvement in the model's performance and the precision of its outputs.

Looking ahead

This approach of task classification using ChatGPT has been a game-changer for us, allowing us to streamline and automate customer support in a way that was previously "hard coded". The insights we've gathered so far are just the beginning. As we continue to fine-tune and expand our task classification, we're excited about the prospects this holds for improving the customer experience.

Our journey underscores not only the incredible potential that AI, and specifically ChatGPT, holds but also how much technique is involved with prompt engineering. While we still have a long way to go, we're excited about the path we're on and look forward to sharing more about our adventures with ChatGPT in the future.

Keep watching this space for more insights and developments. AI-powered customer support is just getting started, and we're thrilled to be a part of this journey!