Are We Ready for AI Health Coaches?

AI-powered health coaching is no longer a vision of the future—it's here. From Apple and Oura's recent announcements to Google, WHOOP, Samsung, and ThriveAI, major companies are introducing AI tools designed to transform how we manage our well-being. These AI systems promise greater accessibility to quality health guidance, personalized insights, and proactive health recommendations. If you're not already using a health coach, you will likely be by the year's end.

But are we truly ready to entrust our health to AI? AI has the potential to be an incredible tool, but only when it is developed and used thoughtfully and responsibly. AI health coaches are no exception. This requires recognizing its limitations, including susceptibility to bias and occasional inaccuracies (commonly referred to as "hallucinations").

Centaur.ai co-founder and CEO Erik Duhaime has this to say on the matter:

Through my work at MIT and Centaur, I've collaborated with leading AI teams to ensure quality at every step of the AI product cycle, from training to model evaluation. Model evaluation and monitoring are critical to AI health coach adoption, guaranteeing accurate, reliable, and safe recommendations while uncovering potential biases for specific patient groups. — Erik Duhaime, CEO, Centaur.ai

AI Health Coaches on the Rise

Recent advancements in AI have paved the way for health assistants to integrate seamlessly into everyday life. Here are a few examples making headlines:

Apple and Samsung Devices

Apple's Health app and Samsung's AI-assisted fitness trackers enhance fitness monitoring and analysis. These systems interpret user data, translate complex metrics into digestible insights, and even nudge users toward healthier behaviors.

Oura Advisor

The Oura Ring now includes an AI tool called Oura Advisor, which acts like a personal coach. Beyond basic data like sleep and activity scores, it provides deeper analysis and actionable recommendations tailored to users' habits and trends. For example, it might review your nighttime data and suggest adjustments to your routine to improve the following night's sleep.

Whoop Coach

Whoop now offers a GPT-4-powered coach feature with its 4.0 fitness band that offers personalized, conversational insights. This helps make complex biometric data like recovery and strain understandable and actionable for users. A recent ZDNet review states that the Whoop Coach empowers users to make effective, real-time decisions to improve their health and fitness.

ThriveAI

Thrive Global has partnered with OpenAI to create the ThriveAI health coach, which delivers hyper-personalized guidance. The ThriveAI health coach learns from users' daily habits, recommending routines like taking a walk after picking up your child or starting a calming bedtime ritual before an early morning flight. This dynamic coaching creates an experience far more immersive than static apps.

These innovations are part of a growing $7 billion health coaching industry. AI health coaches promise accessibility and convenience, but they also prompt a critical question: Are consumers actually benefiting?

Weighing the Benefits and Risks

Like traditional health coaches, AI-powered versions can be highly effective when used responsibly. However, there are benefits and risks worth examining.

Potential Benefits

1. Accessibility: AI health coaches offer 24/7 support without the need for costly, in-person consultations. In addition, AI health coaches will respond more quickly and thoroughly to your MyChart messages than your PCP. AI health coaches meet users where they are—whether through wearables, mobile apps, or desktop platforms.
2. Personalization: AI systems leverage data collected over time, tailoring guidance to individual behaviors and health goals. For example, they can identify subtle patterns, such as a change in heart rate trends, long before users notice these patterns themselves.
3. Consistency: Unlike human coaches, AI doesn't tire, have off days, or show bias in one-on-one interactions (assuming it's designed responsibly). This allows for consistent feedback every time.

Risks and Challenges

1. Bias in Algorithms: AI is only as unbiased as the dataset it's trained on. If training data skews toward one demographic (e.g., younger individuals from urban areas), recommendations for underrepresented groups may be flawed or even harmful.
2. AI Hallucination: Generative AI models are prone to hallucination, a phenomenon where they provide misleading or nonsensical outputs. For example, an AI health coach might misinterpret symptoms or make unreliable dietary suggestions based on incomplete information.
3. Trust and Reliability: Human oversight is essential to ensure these tools function ethically and meet consumer safety standards. Without responsible development, even subtle errors could jeopardize users' health outcomes.

Ensuring Accurate, Safe, and Reliable Guidance

For AI health coaches to thrive, it's imperative to emphasize ethical development and rigorous model evaluation. This includes building safeguards against bias, monitoring model performance, and ensuring accurate recommendations. Here are a few key strategies to help accomplish these goals:

Anchor the Model in High-Quality Data

AI mustn't get key clinical concepts wrong when it responds to a patient. The way to reduce hallucinations is to use an RAG model, so the model is focused on key concepts and grounded in reality/research. OpenEvidence and Consensus.app attempts this by grounding things directly in the scientific literature, but this isn't necessarily the best approach for consumers.

We've seen clients building health coaches and chatbots build out specific databases of clinical concepts (e.g., "type 2 diabetes management" or "nausea"). Still, they need to have subject matter experts refine and QC that database to ensure the articles on each concept are accurate and remain up-to-date, since the standard of care never stops evolving. Otherwise, the model is at risk of spitting out misinformation every time that topic comes up.

Conduct Rigorous Model Evaluation

Accuracy and safety don't end at deployment. Continuous evaluation ensures AI systems adapt to evolving real-world scenarios and consumer needs. Conducting regular evaluations during training and after deployment is crucial to mitigate this. This helps to identify and address any biases promptly, ensuring the model remains fair and effective. For instance, models should be assessed for:

• Accuracy: Do predictions match outcomes across diverse user groups?
• Bias: Are specific demographics disproportionately affected by errors?
• Performance over Time: Does the tool maintain quality as its dataset grows or changes?

Companies like Datadog and Domino offer tools and processes to evaluate ML model performance, form insights, and provide alerts when production models have degraded.

Conduct Continual Model Monitoring

Centaur's Duhaime had this to add about continual model monitoring:

Just because a model works well when deployed does not mean it can be trusted forever, especially when patient well-being is at risk. To tackle this, we work with companies that leverage Centaur's network of subject matter experts to conduct evaluations on an ongoing basis. It's essential to catch situations where a model might give bad advice, even if it passed all the original evaluations. One chatbot we evaluated, for example, told a patient with a physical disability to go for a jog to lose weight—not exactly helpful, and understandably upsetting. No matter how carefully you test a model, things can still go wrong in the real world. And as the world changes—think of trends like the Tide Pod challenge—a health coach might not have the latest guidance, which could lead to dangerous advice. That's why continual model monitoring isn't just a nice-to-have—it's absolutely necessary. — Erik Duhaime, CEO, Centaur.ai

What's Next for AI in Health?

The potential of AI health coaches to transform wellness is immense, but this transformation must be approached with care. Accountability, transparency, and continuous improvement are vital as consumers increasingly rely on these tools to inform critical health decisions.

Although AI health coaching is undeniably promising, it won't replace human professionals. Instead, it is a powerful supplement to traditional healthcare, offering personalized insights and proactive guidance. By blending human expertise with advanced AI solutions, companies can unlock new levels of accessibility, health equity, and innovation.