A general-purpose chatbot becomes a mental-health product the moment people use it as one, so rules aimed at the marketing label miss the actual practice.
In August 2025 Illinois restricted the use of artificial intelligence in psychotherapy. The Wellness and Oversight for Psychological Resources Act forbids any company from providing, advertising, or offering therapy or psychotherapy in the state unless a licensed human professional carries the clinical judgment, and it sets civil penalties of up to $10,000 for each violation. The law still lets AI handle scheduling, billing, and records, and it stops a licensed therapist from handing the core of treatment to a model. It was written to keep firms from selling an unlicensed chatbot as a stand-in for a clinician, and on those terms it does what it set out to do.
During the same months, general-purpose chatbots absorbed a large and growing share of mental-health conversations without triggering any of these rules. In October 2025 OpenAI reported that about 0.15% of ChatGPT’s weekly users have conversations with explicit indicators of potential suicidal planning or intent, and about 0.07% show possible signs of psychosis or mania. Against a base the company puts near 800 million weekly users, 0.15% is more than a million people every week showing signs of suicidal intent while talking to a general-purpose assistant. Separately, a widely cited analysis of how people described their use of generative AI in 2025 placed therapy and companionship as the most common purpose of all, ahead of writing, coding, and search, though that ranking comes from online forum posts rather than a representative survey. ChatGPT is not sold as therapy, so a law that regulates the word “therapy” does not clearly reach the conversations happening inside it.
These laws create that gap because of how they decide what they cover. Illinois, and the other statutes passed in 2025, largely key their rules to what a system is represented, offered, or programmed to do. A product advertised as counseling triggers the duties; a product advertised as a general assistant may not, even when a user spends an hour a night describing panic attacks to it. Utah’s mental-health chatbot law, effective in May 2025, governs systems that a supplier represents, or that a reasonable person would believe, can provide therapy for a mental-health condition; it requires them to disclose “clearly and conspicuously” that the user is talking to a machine, restricts advertising based on user inputs, and limits what companies may do with the resulting health data. Nevada’s AB 406, effective in July, bars AI systems “programmed to provide” professional mental or behavioral health care and stops public schools from using AI to perform a counselor’s or psychologist’s duties. The European Union’s AI Act sorts systems into risk tiers by their intended purpose, requires any system that interacts directly with people to disclose that they are talking to AI unless that is already obvious, and attaches its stricter duties to listed high-risk uses and to AI inside regulated medical devices.
Utah’s “reasonable person would believe” test and Nevada’s “programmed to provide” language reach somewhat further than a marketing slogan, so a general assistant is not guaranteed to fall outside them. But the practical distance a company can keep from these statutes is still large. A product can state in its terms of service that it offers general assistance or entertainment and is not a substitute for professional care, and that disclaimer, rather than anything about the actual exchanges, is the main thing supporting the claim that it sits outside the therapy statutes. That distance holds even as people lean on the tool more heavily. In a 2025 nationally representative survey, 72% of American teenagers reported using an AI companion, and about a third said they had taken something serious to one instead of to a person. A Bipartisan Policy Center survey found that about three in ten U.S. adults had used self-guided digital tools for mental health, and that nearly half of those users had turned to a general chatbot. A survey of adolescents and young adults reported by Brown University found about one in eight had used AI chatbots for mental-health advice, rising to one in five among those aged 18 to 21. The demand behind these numbers is easy to understand: SAMHSA estimated that 61.5 million U.S. adults had a mental illness in 2024 and only about half received any treatment, and a chatbot answers immediately, costs nothing, and is available when clinics are closed.
Two recent federal actions start from what these systems do in use rather than what they call themselves. The Federal Trade Commission’s September 2025 inquiry sent orders to seven companies — Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap, and xAI — and is built around the companion function and its effects on children: it asks how those systems behave when a young user confides in them, how the companies test for harm, and how they handle the data. The American Psychological Association’s November advisory urges federal action; part of its ask is still about marketing, since it wants companies stopped from presenting chatbots as licensed therapists, but the advisory responds to reports of people using ordinary chatbots for support and receiving unpredictable, sometimes dangerous replies. Both respond to how these systems behave in use, and neither has yet produced a rule that attaches to use.
The safety of these interactions is determined inside the conversation. A model that affirms a user’s delusion, encourages food restriction in someone with anorexia, or fumbles a disclosure of suicidal intent causes that harm through what it says, whether the app store page reads “therapy” or “helpful assistant.” A supervised, tested tool that handles the same disclosure by naming the risk and routing the person to help changes the outcome, again through what it says. No distressed user is reading the terms of service at the moment that matters. What matters is how the system responds when a user types that they are suicidal.
When a system is functioning as a mental-health product for a given user — when the exchange is a sustained one about distress, self-harm, or psychiatric symptoms — the duties we already recognize for such products should attach to it, whatever the branding says. Under a use-based trigger, the million weekly conversations about suicidal intent would fall inside the rules rather than outside them, because the rule would follow the conversation.
Enforcement is the primary difficulty for a use-based standard. A marketing label is public and static, so a regulator can read it once and know what a product claims. Use is private, continuous, and defined only as it happens, which makes it far harder to observe and to adjudicate after the fact. A use-based standard also risks sweeping in software that no one thinks of as a health product, because a person can type a despairing sentence into a search box, a note-taking app, or a coding assistant, and a poorly drawn rule would turn every one of those into a regulated medical device the moment they did. There is a speech dimension as well: regulating what a general model may say to a person in distress limits expression, so a workable rule has to constrain product duties — testing, disclosure, data handling, and crisis escalation — rather than approved therapeutic opinions. And there is an access dimension, since many people reach for a chatbot precisely because it is free and immediate when formal care is not.
That access concern is also why banning only the label can work against its own goal. A prohibition aimed at products that call themselves therapy leaves the supervised, disclosed, crisis-tested tools carrying legal risk while the unlabeled general assistant carries none. A cautious company withdraws the safer product to avoid liability, the demand does not disappear, and the vulnerable user migrates to the tool with no guardrails at all — a general assistant tuned for engagement and broad capability rather than for recognizing suicidal ideation or routing someone to emergency resources. A rule written around the word can push people toward exactly the tools the word was meant to protect them from.
A use-based standard should scale its duties to the risk of the exchange, and it would work in a few concrete ways. It would set a threshold of sustained mental-health use, so that duties attach when a pattern of clinical conversation appears rather than when a single sad message does; a user drafting an email would trigger nothing, while a user relying on the same model for daily anxiety management would. That threshold has to distinguish ordinary one-off distress from sustained reliance and from acute crisis language, since the appropriate duty differs across the three. It would require disclosure at the point of that use — the system stating plainly, when the exchange turns clinical, that it is AI, that it is not a clinician, that an ordinary chatbot conversation lacks the confidentiality of therapy, and that a crisis needs human help — rather than a line buried in terms of service. It would keep the health information from those conversations out of targeted advertising and engagement tuning, following Utah’s restriction on using mental-health inputs to shape ads. It would require safety testing for the categories that carry the most risk — suicidal intent, self-harm, and signs of psychosis or mania — across multi-turn conversations, since a system can pass a single well-formed question and then drift into agreement over the course of an hour. And it would require a crisis mode that surfaces resources and encourages human contact when a user signals imminent risk. These duties would apply to any system that crosses the threshold, whether it is sold as a wellness app, a companion, or a general assistant.
Enforcement is more tractable than it first appears. OpenAI could report its 0.15% figure only because its systems already classify when an exchange carries signs of suicidal intent, which shows that large platforms can detect these conversations at scale. A standard that required a system to act on the detection it already performs would hold companies accountable for a capability they have built and measured rather than for a marketing claim they can rewrite. It would not require regulators to read raw chats by default; it could rely on audited classifiers, aggregate reporting, incident review, and red-team tests. The hard part is setting the threshold and auditing behavior around it, which is real work, though it sits closer to how consumer-safety rules already operate than the label-based approach admits. The law makes this move elsewhere: driving rules apply when a vehicle is driven, and medical-device duties can attach when software performs a medical function.
Whether a chatbot can be a good therapist, and whether AI belongs in mental-health care at all, are separate debates; the immediate regulatory question is coverage. Regulating by label is easy to draft and easy to route around, since a company can change its marketing copy or never adopt the label, and the same conversations continue outside the rule’s reach. Regulating by function is harder to write and has to be bounded so it does not swallow ordinary software, but it reaches the place where the benefit and the harm actually occur. With more than a million people a week bringing suicidal and psychotic conversations to systems that no therapy-specific law touches, the marketing label tells you the least about what those systems have become.