Without proper guardrails, artificial intelligence runs the risk of replicating human bias and limiting broad benefits. As the healthcare industry increasingly integrates AI, researchers have the opportunity to curb bias by ensuring their studies include often-overlooked communities and benefit those in need.
But when researchers attempt to diversify their study participants, they can encounter assumptions about which communities are “ready” for innovative health interventions.
The “Diabetes and Mental Health Adaptive Notification Tracking and Evaluation” (DIAMANTE) study recently tested the efficacy of text messages—shaped by a machine learning algorithm—to encourage study participants to take a daily walk and increase physical activity. Study participants included a large percentage of people of color and people from low-income backgrounds, who were diagnosed with diabetes and depression symptoms. Participants were recruited from the San Francisco Health Network and online. Some participants received customized, machine learning-driven texts, while others received random text message reminders or a weekly mood-related text message.
Associate Professor Adrian Aguilera serves as the co-Principal Investigator of the DIAMANTE study. Dr. Aguilera shares what the study means for the burgeoning application of AI to healthcare settings—especially for marginalized communities:
What makes the DIAMANTE study unique?
The DIAMANTE study isn’t the first study using these machine learning methods, but the latest innovations tend to first be applied in higher-resourced settings—it’s easier to reach prospective study participants who are relatively more educated, as they tend to be more responsive and understand the research. We’re still on the cutting edge of looking at AI and machine learning and how those methods improve health interventions. The DIAMANTE study is the first to test a personalized, health-related text message approach in a large, diverse, and multilingual sample.
The DIAMANTE study focuses on both physical (diabetes) and mental health (depression) conditions. What do diabetes and depression have in common?
People in marginalized communities tend to have more comorbidities: When people struggle with poor health, it’s often not just one issue; it may be diabetes and chronic pain and depression. People don’t see their health as a single diagnosis, even though on the clinical side we see it diagnostically.
Additionally, taking a physical approach to health doesn’t feel as stigmatizing as other interventions might. “Behavioral activation” is going out in the world and doing pleasurable things. Physical activity is one proxy for that and can address depression, because when you’re out in the world, you’re not isolating at home. We also know physical activity is a core element of improving overall health, and has physiological benefits in addressing diabetes. Finally, taking a walk every day is straightforward and easy to understand.
Behavior change is notoriously difficult. What worked to motivate the study participants?
We know there are three factors related to behavior change: Capability, opportunity, and motivation to engage in the behavior. In this study, we developed text messages that focused on each of those factors: Can you do this? When can you do this? Why do you want to be healthy?
We deployed machine learning for the adaptive messaging group, so they received text messages from an algorithm that was learning—in real time—the most effective pairings of messages, as well as the best times of day to deploy the messages. The participants who received the adaptive text messages increased their daily steps by 19%. Participants who received a weekly reminder monitoring their mood increased their step count by 3.9%, and the group that received random text messages (not optimized by artificial intelligence) increased their steps by only 1.6%.
How did you leverage machine learning in this study?
We utilized a machine learning approach called “reinforcement learning” which uses step count data collected from mobile phones to “learn” which messages, timing, and ordering will result in increased activity. The model takes data from all participants and delivers theoretically-grounded content in an effective manner. The algorithms helped us figure out, for instance, the most effective pairing and timing of messages. We also include individual-level data—if a participant walked yesterday, are they likely to walk today?—to improve personalization and outcomes.
What are the implications from the DIAMANTE study?
First, we are showing that machine learning algorithms can improve physical activity interventions and second, these algorithms can be applied in broad settings, including with non-English speakers and people from low-income backgrounds.
As we design our studies, we’re navigating this bias built in on all sides—from developers, from health care providers. We face the challenge of people thinking certain interventions are “too high-tech” for Medicaid patients. Yet this study revealed better outcomes and increased physical activity for vulnerable populations.
Oftentimes, there’s a gap between doing this research and having the capacity to connect with communities with high unmet needs. As researchers, we need to put in the work to bridge that gap—for instance, we’re fortunate to have Spanish-speaking folks on our team who can develop recruitment materials. This study teaches us that, if researchers are willing to make the effort, innovative methods can work for all, including populations with unmet needs—the people who usually don’t receive the most innovative care.