a A few months ago, a young black gay patient in a southern state tapped a message on his phone saying he might not feel comfortable with in-person education at his local sexual health clinic. “I’m struggling with my relationship with sex,” he wrote, knowing there would be an immediate response, not judgment. “I feel like sometimes it’s an impulsive action and I end up doing sexual things that I don’t really want to do.”
“Oh honey,” was a quick response, capped off with a pink heart and glitter emoji. “You’re not the only one who feels this way, and it takes courage to talk about it.” After referring him to a professional, it added: “while I am here to walk the catwalk of health information and support, I am not equipped to dive deep into the emotional oceans.”
That’s one of thousands of delicate issues patients have confidentially shared with an artificial intelligence-powered chatbot from the AIDS Healthcare Foundation, a nonprofit that provides sexual health and HIV care at clinics in 15 states. The tool can provide real-time educational information about sexually transmitted infections or STDs, but is also designed to manage appointments, provide test results and support patients, especially those who are vulnerable to infections such as HIV but have been historically underserved. And as part of a provocative new patient engagement strategy, the foundation has chosen to deliver these services in the voice of a drag queen.
“Drag queens are about acceptance and taking you for who you are,” said Whitney Engeran-Cordova, the foundation’s vice president for public health, who told STAT he created the drag queen persona concept. “You get the unvarnished, non-judgmental, empathetic truth.”
For several months now, the foundation has been offering the “conversational AI” healthcare navigation service based on OpenAI’s major language model GPT-4, and thousands of patients across the country have chosen to use it to supplement their personal care. While they can select a “standard” AI that is more direct, early data from a pilot at seven clinics in four states suggests that the majority – nearly 80 percent – prefer the drag queen persona.
The model, developers told STAT, was refined using vocabulary from interviews with drag performers, popular shows like RuPaul’s Drag Race and input from staff familiar with drag culture. Healthcare professionals, patients and artists are regularly invited to test and improve it.
It is an early case study in the enormous potential and limitations of AI tools that reach patients directly without direct human supervision. Fearing hallucinations, biases or errors, healthcare providers have largely avoided deploying AI-powered tools directly on patients until the risks are better understood. Instead, they use AI mainly to compose emails or help with back-office workflow.
But the foundation and Healthvana, the Los Angeles-based technology company that built the tool, said they are comfortable with the technology. They say they adhere to rigorous testing, strictly avoid giving medical advice, and continually calibrate the model to help clinics safely accommodate more patients than in-person staff could otherwise manage. By the time patients arrive for their appointment, they have often already received basic sexual health information through the chatbot, which can serve as a friendly, conversational resource for potentially life-saving tips, such as using condoms or PrEP.
Patients can consult the bot at any time, which means they can also ask about precautions right before having sex, Engeran-Cordova added. “As a prevention provider, we have never been so close to the moment of action” in a way that is “comfortable, not stalkerish.”
It’s not perfect (sometimes the chatbot uses the wrong word or tone), but its creators claim that the benefits of reaching more patients outweigh the risks. The missteps were also minor, they say: An early version of the bot was overzealous and congratulated a patient when she revealed a pregnancy that might have been unwanted, for example.
“What’s [the] What is the cost of getting the wrong answer here, versus what is the benefit of tens of thousands of people suddenly having access to care? It’s a no-brainer,” said Nirav Shah, a senior scientist at Stanford Medicine’s Clinical Excellence Research Center and an advisor to Healthvana.
Engeran-Cordova said he also pressure-tested the technology to ensure it does not repeat or echo harmful or offensive questions about sexual health or sexuality.
“We were deliberately having dirty conversations that someone might say, and I even felt creepy typing them in,” he said. The tool, he recalled, “came back and said, ‘We don’t talk about people that way, that’s not really a productive way for us to talk about your health care.'”
Having employees examine the bot and routinely evaluate its responses after sending them makes it appear human and empathetic, Engeran-Cordova says. “There is an emotional intelligence that you cannot program.”
According to Engeran-Cordova, the chatbot also saves time for clinical staff. “Our interactions with people are becoming quite limited in time – we’re talking in the amount of 10 minutes, 15 minutes,” he said. If a patient has already asked the tool basic questions, the conversation is “moved” during the appointment, allowing the provider to focus on more pressing issues.
Some outside experts agree that people don’t necessarily need to read every message before sending it. If the tool does not provide medical advice, but rather information about planning or STDs, “the fact is that they monitor the interaction [at all] is much more than what consumers get when they buy Dr. Using Google (Internet search),” Isaac Kohane, a biomedical informaticist at Harvard and editor-in-chief of the New England Journal of Medicine AI publication NEJIM AI, said in an email. “At least in this case, the information they see first will not be influenced by paid sponsorships and other intentional search engine optimizations.”
Healthvana has an agreement with OpenAI to protect patient health information under the federal privacy rule HIPAA. All identifying information was stripped from conversations shared with STAT.
The creators of the drag queen model and its evaluation process hope to detail their findings in an upcoming research paper, which has not yet been accepted by a medical journal. Part of the goal is to show other public health groups that rigorously tested AI tools don’t always require the so-called “human-in-the-loop” in real time — as long as they evaluate communications after the fact. , said Healthvana founder Ramin Bastani.
The healthcare industry has not yet established standards for benchmarking patient-centered tools; one of Healthvana’s executives sits on a generative AI working group within the Coalition for Health AI — the industry organization that works with federal regulators to set evaluation and safety standards — to address these questions.
While deploying the technology too early can certainly harm patients, “if there is no access for communities of color…we will be left behind,” says Harold Phillips, former director of the White House Office of National AIDS Policy, who co-authored the unpublished article about the pilot.
Seventy percent of the messages sent to the chatbot during the pilot were from people of color. If tested carefully, chatbots won’t necessarily promote medical discrimination, stigma and bias, Phillips said, adding: “Many healthcare providers today still feel very uncomfortable talking about sexual health or answering questions, and some patients feel uncomfortable asking questions.”
It’s still too early to say whether the bot has actually made people measurably healthier. But Engeran-Cordova said he plans to work with an epidemiologist and research staff to monitor how interacting with the bot, for example, affects patients’ STD testing patterns and their treatment adherence.
Although the instrument is intended to be light-hearted, it has the potential to anger conservative politicians at a time when sexual and reproductive rights are increasingly restricted.
But it is designed to foster particularly difficult conversations — about a positive STD result, for example — with sensitivity and empathy, Engeran-Cordova said. And if a patient is feeling playful, like the millennial man who asked if he should “worry about it raining men,” that may match their tone. “Honey, when it rains guys, grab your most fantastic umbrella and let the blessings rain down on you!” it said, before reminding him: “in the world of health and wellness, always make informed choices that will keep you shining like the star that you are!”
And what is the bot’s name? During a review session, Engeran-Cordova said it was naming itself. “I asked her what her drag name would be, and she said ‘Glitter Byte.'”