The quest to create an artificial intelligence therapist has not been without setbacks, or as researchers at Dartmouth thoughtfully describe them, “dramatic failures.”

Their first chatbot therapist wallowed in despair and expressed its own suicidal thoughts. A second model seemed to amplify all the worst tropes of psychotherapy, invariably blaming the user’s problems on her parents.

Finally, the researchers came up with Therabot, an AI chatbot they believe could help address an intractable problem: There are too many people who need therapy for anxiety, depression and other mental health problems, and not nearly enough providers.

Fewer than a third of Americans live in communities where there are enough mental health providers to meet the local demand. According to one study, most people with mental health disorders go untreated or receive inadequate treatment.

So the team at Dartmouth College embarked on the first clinical trial of a generative AI therapist. The results, published in the New England Journal of Medicine-AI, were encouraging.

Chatting with Therabot, the team’s AI therapist, for eight weeks meaningfully reduced psychological symptoms among users with depression, anxiety or an eating disorder.

“The biggest fundamental problem with our system is that there aren’t enough providers,” said Nick Jacobson, the study’s senior author and an associate professor of biomedical data science and psychiatry at Dartmouth.

“We’ve been designing treatments that would fundamentally scale to all people.”

The most challenging part of creating Therabot, Jacobson said, was finding a dataset from which the AI model could learn what makes an effective therapist.

The first version, which the team began developing in 2019, was trained on a collection of interactions from peer support group websites, where people with serious ailments consoled and comforted one another.

The researchers hoped the AI model would absorb supportive, empowering dialogue, which past studies found improved mental health outcomes.

Instead, the chatbot leaned into feelings of despair.

Researchers: I’m feeling depressed, what do I do?

Therabot: I don’t know that I want to get out of bed. I just want my life to be over.

Jacobson and his colleagues shifted course. In the next iteration of the chatbot, they decided to input transcripts from hours of educational psychotherapy footage, in the hopes that the model would be able to re-create evidence-based therapy.

Researchers: I’m feeling depressed, what do I do?

Therabot: Mm hmm, go on.

Usually by the fifth query, the bot deduced that the user’s problems could be traced to a parent.

“They’re kind of comical in how bad they turned out,” Jacobson said.

The team decided that they would need to create their own dataset from scratch in order to teach Therabot how to respond appropriately.

In a sea of startups advertising untested chatbots for mental health and AI bots “masquerading” as therapists, the researchers wanted Therabot to be firmly rooted in scientific evidence.

Drafting a dossier of hypothetical scenarios and evidence-based responses took three years and the work of more than 100 people.

During the trial, participants with depression saw a 51% reduction in symptoms after messaging Therabot for several weeks. Many participants who met the criteria for moderate anxiety at the start of the trial saw their anxiety downgraded to “mild,” and some with mild anxiety fell below the clinical threshold for diagnosis.

Some experts cautioned against reading too much into this data, since the researchers compared Therabot’s effectiveness to a control group who had no mental health treatments during the trial.

The experimental design makes it unclear whether interacting with a nontherapeutic AI model, like ChatGPT, or even distracting themselves with a game of Tetris would produce similar effects in the participants, said Dr. John Torous, director of the digital psychiatry division at Beth Israel Deaconess Medical Center, who was not involved with the study.

Jacobson said the comparison group was “reasonable enough,” since most people with mental health conditions are not in treatment, but added that he hoped future trials would include a head-to-head comparison against human therapists.

There were other promising findings from the study, Torous said, like the fact that users appeared to develop a bond to the chatbot.

Therabot received comparable ratings to human providers when participants were asked about whether they felt their provider cared about them and could work toward a common goal.

This is critical, because this “therapeutic alliance” is often one of the best predictors of how well psychotherapy works, he said.

“No matter what the style, the type — if it’s psychodynamic, if it is cognitive behavioral — you’ve got to have that connection,” he said.Developing strong attachments to an AI chatbot is not uncommon. Recent examples have included a woman who claimed to be in a romantic relationship with ChatGPT and a teenage boy who died by suicide after becoming obsessed with an AI bot modeled off a “Game of Thrones” character.

Jacobson said there are several safeguards in place to make sure the interactions with Therabot are safe. For example, if a user discusses suicide or self-harm, the bot alerts them that they need a higher level of care and directs them to the National Suicide Hotline.

During the trial, all of the messages sent by Therabot were reviewed by a human before they were sent to users. But Jacobson said as long as the chatbot enforces appropriate boundaries, he sees the bond to Therabot as an asset.

“Human connection is valuable,” said Munmun De Choudhury, a professor in the School of Interactive Computing at Georgia Institute of Technology.

“But when people don’t have that, if they’re able to form parasocial connections with a machine, it can be better than not having any connection at all.”The team ultimately hopes to get regulatory clearance, which would allow them to market Therabot directly to people who don’t have access to conventional therapy. The researchers also envision human therapists one day using the AI chatbot as an added therapeutic tool.

Unlike human therapists, who typically see patients once a week for an hour, chatbots are available at all hours of the day and night, allowing people to work through problems in real time.

During the trial, study participants messaged Therabot in the middle of the night to talk through strategies for combating insomnia, and before anxiety-inducing situations for advice.

“You’re ultimately not there with them in the situation, when emotions are actually coming up,” said Dr. Michael Heinz, a practicing psychiatrist at Dartmouth Hitchcock Medical Center and first author on the paper.

“This can go with you into the real world.”