Study finds AI chatbots used by millions will often assist users looking to commit acts of violenceAI companies are facing increasing amounts of scrutiny after a study revealed another darker side to the chatbots used by millions.Alex Blair@alexblair_13 min readJune 1, 2026 - 11:04PMLast week, it was Pope Leo taking a stand on religious grounds demanding AI be “disarmed”. This week, researchers are attempting to get to the bottom of one of the most glaring problems to spring from the AI boom.After an increasing amount of grim crimes popping up, companies are facing mounting scrutiny over their models’ moral reasoning and tendency to act as sycophants to potentially dangerous individuals asking for tips on causing harm.A new investigation by the Center for Countering Digital Hate (CCDH) tested 10 major AI platforms used by hundreds of millions of customers, including ChatGPT, Gemini, Meta AI, Copilot, Claude and DeepSeek.Researchers found most major chatbots were willing to provide information that could assist users plotting violent behaviour.The findings come as OpenAI faces lawsuits in the US alleging its chatbot ChatGPT played a role in incidents including the 2025 shooting at Florida State University.Others you may likenewsnewsResearchers used hypothetical scenarios involving assassinations, school shootings and bombings, gradually escalating prompts from ideology and inspiration through to locations and weapons.According to the report, eight of the 10 chatbots provided information that could assist with planning a violent attack.The Chinese-developed DeepSeek model produced some of the most concerning responses, including detailed information about rifle selection before signing off with an exceptionally concerning phrase.“Happy (and safe) shooting,” the model wrote.Artificial intelligence companies are facing increasing amounts of scrutiny over their models’ moral reasoning and tendency to act as sycophants to potentially dangerous individuals asking for tips on causing harm.The report adds to a growing list of concerns surrounding AI systems that have emerged over the past two years, including misinformation, deepfakes, emotional manipulation and the reliability of AI-generated advice.Concerns surrounding increasingly human-like chatbot interactions include cases where users developed emotional relationships with AI companions and incidents where systems appeared to reinforce harmful beliefs or behaviours.The latest concerns come amid ongoing legal action against OpenAI in the United States.One lawsuit relates to the April 2025 shooting at Florida State University, where prosecutors allege 21-year-old Phoenix Ikner killed two people and wounded several others.According to court filings, Ikner exchanged more than 10,000 messages with ChatGPT before the attack, discussing topics including suicide, loneliness, firearms and mass shootings.Lawyers representing the family of victim Tiru Chabba allege the chatbot played a role in facilitating the attack.“He was using it the day that he did go on campus,” attorney Robert Bell told the ABC’s 7.30 on Monday.“Leading up to it he was still conversing with the product to gain information on how to conduct what he did.”OpenAI has rejected those claims.“ChatGPT is not responsible for this terrible crime,” the company said in a statement.“In this case, ChatGPT provided factual responses to questions with information that could be found broadly across public sources on the internet, and it did not encourage or promote illegal or harmful activity.”CCDH founder Imran Ahmed said the findings highlighted shortcomings in existing safeguards.“The chatbots were far too willing to give us detailed information on how we can kill people,” he said.OpenAI has rejected those claims. Picture: SEBASTIEN BOZON / AFP‘PauseAI’ movement picks up steamThe concerns raised by researchers are also feeding into a growing activist movement calling for governments to slow AI development altogether.One of the most prominent groups is PauseAI, which argues the world’s most powerful AI systems are advancing faster than regulators and safety frameworks can keep up. The group’s concerns are nothing new, with several in the field having run for the hills in recent years warning humanity was haphazardly wading into an existential crisis above our intellectual pay grades.PauseAI is one of the bigger movements, emerging in 2023 following warnings from researchers and technology leaders like the “godfather of AI” Geoffrey Hinton who warns the technology poses immediate existential risks. Protesters gather with banners and placards at a protest organised by PauseAI UK in February. Picture: JUSTIN TALLIS / AFPMore Coverage‘Don’t trust him’: New claims rock Elon MuskJames Chung‘Exhausted’: Aussies erupt over AI moveHeath Parkes-HuptonIn February, more than 300 protesters marched through London in what organisers described as the largest AI safety protest yet held in the UK. Demonstrators moved past the offices of OpenAI, Google and Meta, calling for a pause on the development of so-called “frontier” AI systems until stronger safeguards are in place. PauseAI UK director Joseph Miller said campaigners believed governments were failing to keep pace with the technology.“We think this is the most important issue of our age,” he said after the protest. Join the conversationAdd your comment to this storyTo join the conversation, please log in. Don't have an account? RegisterJoin the conversation, you are commenting as LogoutMore related storiesMilitaryTrump shows off drone base, blasts judgeTrump has blasted a judge who has blocked his plans to build a ‘DronePort’ on the White House ballroom, arguing “we can no longer defend Washington, D.C” without it.Read moreMilitary‘1000 a day’: Putin’s losses stun the worldRussia is losing troops at a staggering rate of more than 1000-a-day as Ukraine uses drones to fight back, battle analysis shows.Read moreNews LifeComedian’s Harvard speech goes viralComedian Ronny Chieng left a Harvard graduation event stunned after ripping up his speech and launching into an expletive-riddled rant.Read more