There is a startup in New Zealand that already knows when you are in crisis. If you have ever told ChatGPT, Claude, or Gemini something that suggested you might hurt yourself, ThroughLine is the company that decided what happened next. They redirect at-risk users to human crisis counselors and chatbot-based support for self-harm, domestic violence, and eating disorders.

Now they want to do the same thing for people who are radicalizing toward violence.

According to Reuters, ThroughLine founder Elliot Taylor, a former youth worker, is developing a tool that would route users showing violent extremist tendencies to human and chatbot-based deradicalization support. The initiative comes after a growing wave of lawsuits accusing AI companies of failing to prevent, and in some cases enabling, real-world violence.

The trigger was specific. In February, the Canadian government threatened to intervene after OpenAI revealed that a person who carried out a deadly school shooting had been previously banned from ChatGPT, but the authorities were never informed. That incident exposed a gap that nobody in Silicon Valley wants to talk about: AI companies know when users are dangerous, but they have no obligation to tell anyone.

ThroughLine's existing product is already embedded across the three biggest AI platforms. OpenAI, Anthropic, and Google all contract with them. Expanding from self-harm intervention to extremism prevention is a logical next step, but a far more complex one. Talking someone out of hurting themselves is fundamentally different from talking someone out of an ideology.

The harder question is whether this should be a startup's job at all. Deradicalization is traditionally the domain of intelligence agencies, law enforcement, and specialized nonprofits with years of expertise. Handing it to a contractor that also handles eating disorder referrals raises serious questions about competence and scope creep.

But the alternative is worse. Right now, if someone spends weeks telling an AI chatbot about their plans to commit violence, the chatbot might refuse to help. It might add a disclaimer. It might flag the conversation internally. What it almost certainly will not do is call the police. ThroughLine is at least trying to build the bridge between detection and intervention. Whether they are the right ones to build it is a separate conversation.

First reported by Reuters.