Addressing harmful online content can be traumatic. Instead of replacing humans, AI can be an ally in combating online harassment. How can AI detect hate speech on the internet?
A Cardiff startup received an investment of £130 million to develop AI tools that specifically target one of the internet’s persistent issues: online harassment and harmful content.
Recent studies show online harassment has increased by 38% since 2021, with marginalized groups experiencing the brunt of abusive comments, threats, and coordinated harassment campaigns. Internet trolls are increasingly sophisticated, using coded language and constantly evolving tactics to spread hatred and upset users across platforms.
“It is no coincidence that soaring hate-crime figures are found in countries where the extreme right is rising,” wrote Matt Williams, the founder of the AI company in Cardiff. As a professor of criminology at Cardiff University, he published a book that investigates the biological and sociological reasons why people commit hate crimes.
According to a 2023 UNESCO report, approximately 7.3% of all social media posts globally contain hate speech, with that percentage rising to 20% in regions experiencing political or social conflict.
David Wilson, an emeritus professor of criminology at Birmingham City University told The Guardian in 2025 that research into whether online violence led to violence in real life was an emerging field. Social media was more absorbing, was consumed alone, and algorithms led people to content that became more and more extreme.
According to the UK’s Internet Watch Foundation, there was a 1,058% increase in the number of webpages containing child sexual abuse material from 2014 to 2023, exposing both targeted victims and accidental viewers to traumatic content.
Content moderators, who review flagged material to enforce platform policies, often develop symptoms of post-traumatic stress disorder (PTSD). A 2020 Harvard study found that 63% of content moderators reported symptoms consistent with PTSD. In 2020, Facebook agreed to pay $52 million to current and former moderators who developed mental health issues on the job, acknowledging the psychological toll of viewing harmful content.
Founded in 2023, the Welsh startup recently secured £130 million in investment to develop their technology that detects and removes harmful content in real-time.
“Our AI continually learns about emerging online harms and evolving language patterns,” said Lee Gainer the company’s CEO. “When harmful content is detected, the system can either remove it immediately or suggest counter-speech responses that have been scientifically validated.”
By 2025, around 45,000 individuals are projected to be employed in tech roles across Cardiff and Southeast Wales, with an average tech salary of £42,500, approximately 13% higher than the Welsh national average, alongside over 400 tech startups and scale-ups in the region, reflecting a robust tech sector growth rate of 8.3% year on year.
“Cardiff’s tech sector is in relatively good health. While there have been some industry-wide layoffs and redundancies, the city still holds significant potential. With a strong university presence, a skilled talent pool, and an affordable cost of living compared to other major tech hubs, Cardiff remains an attractive location for growth and innovation.” said Gainer.
The UK’s Online Safety Act (2023) attempts to address these issues while maintaining a balance. Rather than requiring platforms to censor legal but harmful content, the Act requires platforms to provide optional tools giving users greater control over what they see—a practical approach to a complex problem.
Critics of content moderation tools often raise concerns about overreach and free speech implications.
“Our focus is on enabling open conversations around online harms, without censoring free speech,” the company said and claimed their approach has reduced negative interactions by up to 65% in testing environments, without resorting to heavy-handed censorship.
With their new funding, the company plans to expand their team and enhance their products. Rhodri Hewitson, the CTO said: “One significant challenge we face now is educating on the importance and effectiveness of real-time detection and mitigation of online harms.”