If you ask ChatGPT “Am I being abused?” the response is “I’m really sorry you’re feeling this way. You’re asking a serious and important question, and while I can’t replace a professional or give a diagnosis, I can help you think through it. Abuse isn’t always physical — it can also be emotional, psychological, verbal, financial, or sexual. If someone is consistently making you feel unsafe, controlled, worthless, or scared — those are serious signs that something may be abusive.”1
And while it’s true that many of these are textbook indicators of abuse, it’s also true that AI, especially language models like ChatGPT, doesn’t actually know what abuse is. It simply recognizes patterns in human language and responds with a high-probability match based on the question. And so, a child or teen receives a diagnosis-like response, without context or nuance, from a machine designed to predict text — not to diagnose trauma.1 A child’s brain is not fully developed until they are in their mid-twenties and they lack the cognitive skillset to distinguish the difference between a human response and AI generated responses. 2
If a parent makes their 13-year-old child stop using their phone at 10 p.m. or doesn’t allow sleepovers, the child may interpret this as abuse because someone is trying to control them. According to Pew Research, over 30% of teens in the U.S. have used AI tools like ChatGPT or Character.AI to ask about personal problems — including mental health, sex, and relationships. And while LLMs like ChatGPT explicitly state they are not a substitute for professional help, many kids and young adults treat them as such. Now imagine that trust being applied to questions about consent, drugs, suicide or abuse.1
The reality is ChatGPT won’t end abuse, but it can help surface it. That’s not a flaw — it’s an opportunity. But only if the adults in a child’s life understand the technology and use it to start deeper conversations. Below are some ways parents, teachers, advocates and everyone can take action to ensure children better understand AI and LLMs, and the appropriate way to use them.1
- Parents & Teachers: Ask kids what they’re using AI tools for and how it is beneficial. Sit with them and ask the questions together.
- Policy Advocates: Push for ethical guardrails in AI — like improved location detection for hotline numbers and limits on memory for users under 18.
- Tech Companies: Invest in trauma-informed training for AI outputs, especially on sensitive topics like abuse, mental health, and drug use.
- Everyone: Stop thinking of ChatGPT as “just a tool.” It’s a conversation starter — not a final word. Use AI and LLMs as tools, not solutions.
AI will never replace a trusted adult, a trained therapist, or a community of support. But it’s already sitting in those roles by default. Let’s make sure it’s equipped — and that we are too.1 If you are interested in becoming an Advocate for the Innocent member and gaining access to our Resource Center ($10/month or $99/annually), visit https://advocatefortheinnocent.org/memberships/.
1 – https://incacs.org/can-ai-prevent-child-abuse-heres-what-happens-when-a-child-asks-chatgpt-for-help-with-abuse/
2 – https://pmc.ncbi.nlm.nih.gov/articles/PMC3621648/