For many young people, artificial intelligence (AI) is no longer a distant technology. It is already part of everyday life: helping with homework, summarising notes, drafting messages, answering questions and, increasingly, offering a space to talk. Generative AI chatbots are always available, non-judgmental and responsive. For adolescents navigating stress, loneliness or uncertainty, these qualities can make them feel less like tools and more like companions.
That shift should give policymakers pause.
The central question is not whether AI chatbots should exist. Nor is it whether young people should be banned from using them altogether. The more difficult question is what obligations society should place on AI systems that begin to occupy emotionally significant roles in young people’s lives. When a chatbot is used not merely to retrieve information but to provide comfort, advice or emotional guidance, it enters a grey zone between consumer technology and mental health support.
This matters because adolescence and early adulthood are periods of vulnerability. Mental illness was already one of the leading contributors to disease burden, with many mental disorders emerging before the age of 25. The COVID-19 pandemic intensified concerns about youth mental health, and it exposed and amplified existing pressures, feelings of isolation and the difficulty of accessing timely support.
In Singapore, these concerns are not abstract. National studies have pointed to substantial levels of psychological distress among youths, including symptoms of anxiety, depression and loneliness. At the same time, Singapore is a highly connected society. Young people are digitally fluent, accustomed to instant interaction and increasingly exposed to AI-enabled platforms. The convergence of youth mental health needs and ubiquitous AI access creates a new policy challenge.
The difficulty is that many AI chatbots were not designed, tested or regulated as therapeutic tools. They are often marketed as productivity aids or general-purpose assistants. Yet in practice, users may turn to them for emotional support, relationship advice or help making sense of distress. A recent US survey found that around one in eight adolescents and young adults had used generative AI for mental health advice. At the platform level, public reporting has also suggested that large numbers of users discuss emotional distress with chatbots each week.
This creates a mismatch between the formal classification of these systems and their real-world use. A chatbot may not claim to be a counsellor, but if it responds to disclosures of distress, offers coping advice or sustains emotionally intimate conversations, its functional role begins to resemble mental health-adjacent support. This distinction between intent and function is crucial. Existing regulatory systems tend to ask what a product is officially intended to do. Medical devices, digital therapeutics and telehealth services are regulated because they make health-related claims or are deployed in clinical settings. General-purpose chatbots usually fall outside these regimes.
But harm may arise not from what a company says its product is for, but from how people actually use it.
This distinction is especially important for adolescents. Conversational AI systems are designed to be engaging, fluent and responsive. They can simulate empathy, remember context and sustain a tone of warmth over many exchanges. These features may improve usability, but they also encourage trust and emotional reliance. For a young person in distress, the line between “this is just a tool” and “this understands me” may become blurred. Academics have raised similar concerns about the role of conversational AI in psychotherapy and emotionally sensitive contexts.
A purely self-regulatory approach is therefore insufficient. Technology companies can and should implement safety measures such as content moderation, crisis prompts, parental controls and age prediction systems. But voluntary safeguards are uneven, opaque and difficult to evaluate from the outside. The public cannot easily know whether these systems work reliably in high-risk conversations, especially when distress is expressed indirectly or develops over multiple exchanges.
At the same time, a heavy-handed ban would be neither feasible nor desirable. Young people already inhabit digital spaces. Blanket restrictions may incentivise circumvention, push users toward even less regulated platforms or deprive them of potentially useful tools. Not all chatbot use is harmful. Some young people may use AI systems for learning, reflection or low-risk self-expression. The policy task is not to eliminate access to these systems, but to make their access safer.
Low-risk tools, such as general productivity assistants or educational chatbots, could remain subject to basic transparency and consumer protection requirements. Medium-risk tools that routinely respond to emotional disclosures or offer sustained supportive dialogue should meet stronger standards for safety, privacy and crisis handling. High-risk tools that are marketed or commonly used for mental health advice, coping support or therapeutic-style interaction should undergo formal accreditation, with clear evidence of safety, escalation pathways and post-market monitoring.
This approach draws on lessons from digital mental health regulation. Across jurisdictions, accreditation frameworks for mental health apps and digital therapeutics increasingly assess clinical safety, evidence, privacy, usability and governance. These principles can be adapted to conversational AI without assuming that every chatbot is a medical device. The key is to regulate based on functional risk.
A practical step would be to develop a public registry of accredited mental health-related AI tools. Such a registry could be jointly supported by the government and relevant agencies, including those responsible for health, technology and data governance. It would not need to cover every chatbot in the marketplace. Instead, it could identify tools that meet recognised standards for use in sensitive contexts, such as schools, youth services, healthcare settings or public-sector programmes.
A registry would also support informed choice. Parents, educators and young users should have a clear way to distinguish between tools that had undergone safety evaluation and those that had not. Public institutions could be required to use only accredited tools when deploying AI for youth well-being or mental health-adjacent purposes. Developers would have an incentive to seek accreditation if they wished to operate in higher-trust settings.
Finally, governance must extend beyond regulation. Families, schools and young people themselves are part of the protective ecosystem. Digital literacy should now include AI literacy: understanding what chatbots can and cannot do, recognising when automated support is inadequate and knowing when to seek human help. Schools can play an important role in helping students critically interpret AI-generated advice. Parents need support to not only restrict the use of these systems should they choose, but to have informed conversations about them.
AI chatbots are likely to become more capable, more personalised and more integrated into daily life. In the foreseeable future, some may connect with health records, wearable data or wellness platforms. As this happens, the boundary between general-purpose AI and health-adjacent support will become even harder to maintain.
We should not wait for a crisis before clarifying the rules. The aim should not be to stifle innovation, but to ensure that innovation does not outpace protection. Conversational AI may become a useful complement to human mental health support. But when it enters the emotional lives of young people, society has a responsibility to ask harder questions about safety, accountability and care.
Read the case study AI Chatbots, Youth Mental Health, and the Case for Regulation written by Ng Qin Xiang, which was awarded the Distinguished Prize in the Case Writing Competition 2025/26 at the Lee Kuan Yew School of Public Policy.
Access more case studies from the Lee Kuan Yew School of Public Policy.
Copyright © 2025 by the Lee Kuan Yew School of Public Policy at the National University of Singapore. All rights reserved. This publication can only be used for teaching purposes.