Candidly's AI Predicts Conversation Outcomes to Boost User Engagement
Candidly's innovative approach with Cait allows for a dynamic conversation experience, shifting from after-the-fact evaluations to real-time guidance, significantly enhancing user engagement and satisfaction.
Key Facts
- Candidly's AI model predicts conversation outcomes with 90% accuracy, enhancing user engagement.
- User message length correlates with resolution; longer messages indicate higher engagement potential.
- Engagement states reveal that 78% resolution in high engagement contrasts with 30% in low states.
- Policy shifts reduced disengagement from 23% to 11%, indicating effective real-time conversation management.
- Candidly's approach positions it to serve 1 in 2 U.S. workers, showcasing significant market penetration potential.
Summary
Summary
Candidly, a financial technology company, faced challenges in optimizing their AI financial planner, Cait, to improve user engagement and resolution rates during conversations. By implementing a turn-level state inference model, they enhanced Cait's ability to predict and influence conversation outcomes in real-time. This resulted in a significant reduction in disengaged user interactions, decreasing from 23% to 11%.
Background
Candidly, founded in 2016, operates in the financial technology sector, providing personalized financial guidance to employees through partnerships with employers and financial institutions. Prior to deploying the new model, Cait's performance was primarily assessed after conversations ended, limiting the ability to make real-time adjustments to improve user experience and outcomes.
Challenge
The primary challenge was to enhance Cait's conversational capabilities to ensure users received timely and relevant assistance, thereby reducing the number of conversations that ended without resolution or user engagement.
Solution
Candidly developed a hybrid labeling pipeline to analyze production conversations with Cait, allowing them to predict conversation outcomes based on user engagement signals. They implemented an Input-Output Hidden Markov Model (IO-HMM) to infer user states and guide Cait's responses in real-time. This model utilized features such as message length and topic continuity to assess and influence user engagement during interactions.
Results
The implementation of the turn-level state inference model led to a reduction in the share of user turns in the disengaging state from 23% to 11%. Additionally, the proportion of users in the engaged state increased from 53% to 64%, significantly improving the likelihood of conversation resolution.
Key Insights
- Real-time state inference can transform how conversational agents operate, allowing for immediate adjustments based on user engagement.
- Understanding user behavior patterns is crucial for enhancing AI interactions.
- Evaluating agent performance based on specific engagement states rather than averages provides deeper insights into what strategies are effective.
Customer Testimonial
No direct quotes were provided in the source material.
Entities Mentioned
Companies
Products
Technologies
People
Key Concepts
Definitions
- turn-level state inference
- A method of assessing the current state of a conversation at each turn, allowing for real-time adjustments to improve outcomes.
- Input-Output Hidden Markov Model (IO-HMM)
- A statistical model that separates user-side signals from agent-side features to predict the next state of a conversation.
- conversation analytics
- The process of analyzing conversational data to derive insights and improve interaction outcomes.
- policy surface
- A mapping from the current state of a conversation to the most effective response change to enhance user engagement.
- user engagement states
- Different levels of user involvement in a conversation, which influence the likelihood of achieving resolution.
Use Cases
- →Comparing savings strategies
- →Evaluating repayment options
- →Understanding financial deadlines
- →Deciding on budget adjustments
- →Assessing trade-offs in financial decisions
- →Navigating educational costs
Frequently Asked Questions
What is Cait?
Cait is an AI financial planner developed by Candidly that assists users in making informed financial decisions. It helps compare various financial strategies and provides personalized guidance.
How does Candidly improve conversation outcomes?
Candidly uses turn-level state inference and conversation analytics to assess user engagement in real-time. This allows Cait to adjust its responses dynamically to enhance user experience and resolution rates.
What technologies does Candidly utilize?
Candidly employs AI and machine learning technologies, specifically using models like the Input-Output Hidden Markov Model to analyze user interactions and improve conversational outcomes.
What are user engagement states?
User engagement states are different levels of user involvement during a conversation, which can range from highly engaged to disengaged. These states help predict the likelihood of conversation resolution.
How does Candidly track conversation outcomes?
Candidly tracks conversation outcomes using a hybrid labeling pipeline that analyzes user interactions and applies deterministic rules to assess whether a conversation was resolved or abandoned.