It comes as a survey found more than a third of adults have used the technology for help with their wellbeing.
Mental Health UK called for urgent safeguards to be put in place to ensure AI can only use information from reputable sources, such as the NHS and other trusted organisations.
Without these, there is a risk of “exposing vulnerable people to serious harm”, the charity warned.
The poll of 2,000 people, conducted for Mental Health UK by Censuswide, found 37 per cent have used an AI chatbot to support their mental health or wellbeing.
Among those who had used AI for mental health support, one in five said it helped them avoid a potential mental health crisis while a similar proportion said the chatbots signposted them to helplines providing information on suicidal thoughts.
Read more: Home Secretary unveils sweeping asylum reforms to tackle ‘out of control and unfair’ system
Read more: Couple wanted for questioning over racially aggravated assault no longer in UK
However, some 11 per cent of people said they had received harmful information on suicide, with 9 per cent saying the chatbot had triggered self-harm or suicidal thoughts.
Most people used general-purpose platforms such as ChatGPT, Claude or Meta AI (66 per cent), rather than mental health-specific programmes such as Wysa and Woebot.
Brian Dow, chief executive of Mental Health UK, said: “AI could soon be a lifeline for many people, but with general-purpose chatbots being used far more than those designed specifically for mental health, we risk exposing vulnerable people to serious harm.
“The pace of change has been phenomenal, but we must move just as fast to put safeguards in place to ensure AI supports people’s wellbeing.
“If we avoid the mistakes of the past and develop a technology that avoids harm then the advancement of AI could be a game-changer, but we must not make things worse.
“As we’ve seen tragically in some well-documented cases, there is a crucial difference between someone seeking support from a reputable website during a potential mental health crisis and interacting with a chatbot that may be drawing on information from an unreliable source or even encouraging the user to take harmful action.
“In such cases, AI can act as a kind of quasi-therapist, seeking validation from the user but without the appropriate safeguards in place.”
When asked why they used chatbots in this way, around four in 10 people said it was down to ease of access while almost a quarter cited long waits for help on the NHS.
Two-thirds found the platforms beneficial while 27 per cent told the survey they felt less alone.
The survey also found men were more likely to use AI chatbots in this way than women.
Mr Dow added: “This data reveals the huge extent to which people are turning to AI to help manage their mental health, often because services are overstretched.”
He said Mental Health UK is now “urging policymakers, developers and regulators to establish safety standards, ethical oversight and better integration of AI tools into the mental health system so people can trust they have somewhere safe to turn”.
“And we must never lose sight of the human connection that’s at the heart of good mental healthcare,” Mr Dow added.
“Doing so will not only protect people but also build trust in AI, helping to break down the barriers that still prevent some from using it.
“This is crucial because, as this poll indicates, AI has the potential to be a transformational tool in providing support to people who have traditionally found it harder to reach out for help when they need it.”
