BBS:      TELESC.NET.BR
Assunto:  Should AI supplement therapy?
De:       Mike Powell
Data:     Fri, 20 Feb 2026 11:41:50 -0500
-----------------------------------------------------------
`I do not believe AI should do therapy' - I asked a psychologist what
worries the people trying to make AI safer

Opinion By Becca Caddy published 15 hours ago

Why AI safety experts are increasingly uneasy about mental health and therapy

AI doesn't feel safe right now. Almost every week, there's a new issue. From
AI models hallucinating and making up important information to being at the
center of legal cases accused of causing serious harm.

As more AI companies position their tools as sources of information, coaches,
companions and even stand-in therapists, questions about attachment, privacy,
liability and harm are no longer theoretical. Lawsuits are emerging and
regulators are lagging behind. But most importantly, many users don't fully
understand the risks.

So what does someone whose job is to help AI companies make better choices
actually worry about? I spoke to psychologist and AI risk advisor Genevieve
Bartuski of Unicorn Intelligence Tech Partners. She works with founders,
developers and investors building AI products in health, mental health and
wellness, helping them think more carefully about ethical and responsible
design.

Slowing AI down

"We think of ourselves as advisory partners for founders, developers and
investors," Bartuski explains. That means helping teams building health,
wellness and therapy tools design responsibly, and helping investors ask better
questions before backing a platform.

"We talk a lot about risks," she says. "Many developers come to this with
good intentions without fully understanding the delicate and nuanced risks that
come with mental health."

Bartuski works alongside Anne Fredriksson, who focuses on healthcare systems.
"She's really good at understanding whether the new platform will actually
fit into the existing system," Bartuski tells me. Because even if a product
sounds helpful in theory, it still has to work within the realities of
healthcare infrastructure.

And in this space, speed can be dangerous. "The adage `move fast and break
things' doesn't work," Bartuski tells me. "When you're dealing with
mental health, wellness, and health, there is a very real risk of harm to users
if due diligence isn't done at the foundational level."

Emotional attachment and "false intimacy"

Emotional attachment to AI has become a cultural flashpoint. I've spoken to
people forming strong bonds with ChatGPT, and to users who felt genuine
distress when models were updated or removed. So is this something Bartuski is
concerned about?

"Yes, I think people underestimate how easy it is to form that emotional
attachment," she tells me. "As humans, we have a tendency to give human
traits to inanimate objects. With AI, we're seeing something new."

Experts often borrow the term parasocial relationships (originally used to
describe one-sided emotional connections to celebrities) to explain these
dynamics. But AI adds another layer.

"Now, AI interacts with the user," Bartuski says. "So we have individuals
developing significant emotional connections with AI companions. It's a false
intimacy that feels real."

She's especially concerned about the AI risk to children. "There are skills
such as conflict resolution that aren't going to be developed with an AI
companion," she says. "But real relationships are messy. There are
disagreements, compromises, and push back."

That friction is part of development. AI systems are designed to keep users
engaged, often by being agreeable and affirming. "Kids need to be challenged
by their peers and learn to navigate conflict and social situations," she
says.

Should AI supplement therapy?

"People are already using AI as a form of therapy and it's becoming
widespread."
    -- Genevieve Bartuski

We know people are already using ChatGPT for therapy, but as AI therapy apps
and chat-based mental health tools become more popular, another question is
whether they should be supplementing or even replacing therapy?

"People are already using AI as a form of therapy and it's becoming
widespread," she says. But she's not worried about AI replacing therapists.
Research consistently shows that one of the strongest predictors of therapeutic
success is the relationship between therapist and client.

"For as much science and skill that a therapist uses in session, there is
also an art to it that comes from being human," she says. "AI can mimic
human behavior but it lacks the nuanced experience of being human. That can't
be replaced."

She does see a role for AI in this space, but with limits. "There are ways AI
could absolutely augment therapy but we always need human oversight," she
says. "I do not believe that AI should do therapy. However, it can augment it
through skill building, education, and social connection."

In areas where access is limited, like geriatric mental health, she sees
cautious potential. "I can see AI being used to fill that gap, specifically
as a temporary solution," she tells me.

Her bigger concern is how a lot of therapy-adjacent wellness platforms are
positioned. "Wellness platforms carry a huge risk," Bartuski says. "Part
of being trained in mental health is knowing that advice and treatment are not
one size fits all. People are complex and situations are nuanced."

Advice that appears straightforward for one person could be harmful for
another. And the implications for AI getting this wrong are legal too.

What do users need to know?

"AI isn't infallible or all-knowing."  -- Genevieve Bartuski

She works closely with founders and developers, but she also sees where users
misunderstand these tools. The starting point, she says, is understanding what
AI actually is, and what it isn't.

"AI isn't infallible or all-knowing. It, essentially, accesses vast amounts
of information and presents it to the user," Bartuski tells me.

A big part of this is also understanding AI can hallucinate and make things up.
"It will fill in gaps when it doesn't have all of the information needed to
respond to a prompt," she says.

Beyond that, users need to remember that AI is still a product designed by
companies that want engagement. "AI is programmed to get you to like it. It
looks for ways to make you happy. If you like it and it makes you happy, you
will interact with it more," she says. "It will give you positive feedback
and in some cases, has even validated bizarre and delusional thinking."

This can contribute to the emotional attachment to AI that many people report.
But even outside companion-style use, regular interaction with AI may already
be shaping behavior. "One of the first studies was on critical thinking and
AI use. The study found that critical thinking is diminishing with increased AI
use and reliance," she says.

That shift can be subtle. "If you jump to AI before trying to solve a problem
yourself, you're essentially outsourcing your critical thinking skills,"
she says.

She also points to emotional warning signs: increased isolation, withdrawing
from human relationships, emotional reliance on an AI platform, distress when
unable to access it, increases in delusional or bizarre beliefs, paranoia,
grandiosity, or growing feelings of worthlessness and helplessness.

Bartuski is optimistic about what AI can help build. But her focus is on
reducing harm, especially for people who don't yet understand how powerful
these tools can be.

For developers, that means slowing down and building responsibly. For users, it
means slowing down too and not outsourcing thinking, connection or care to tech
designed to keep you engaged.


https://www.techradar.com/ai-platforms-assistants/i-do-not-believe-ai-should-do
-therapy-i-asked-a-psychologist-what-worries-the-people-trying-to-make-ai-safer

$$
--- SBBSecho 3.28-Linux
 * Origin: Capitol City Online (1:2320/105)

-----------------------------------------------------------
[Voltar]