In August, Austrian authorities arrested three teenagers suspected of planning a mass casualty attack on Taylor Swift’s Vienna concerts, inspired by Islamic State (IS). At least two of the suspects – aged 17 and 19 – were allegedly ‘self-radicalised’ online.

Director of Technology and Society for ISD US Isabelle Frances-Wright, Executive Director for Africa, the Middle East and Asia (AMEA) Moustafa Ayad and Digital Policy Manager for ISD US Ellen Jacobs found that terrorist content is easily accessible to minors using search terms referencing prominent terrorist ideologues.

Platform policies were highly inconsistent, and the inadequacies of their enforcement highlights the continued importance of regulation. Whereas the EU’s Digital Services Act (DSA) mandates greater transparency and data access – many others including the US are yet to enact strong enough safeguards for minors.

Also in this newsletter:

• Explainer: EU Digital Services Act (DSA)
• Online gendered abuse/disinformation during South Africa’s 2024 elections
• Ireland’s far-right using abortion activism to further ethnonationalism
• Research compilation on online antisemitism
• ISD analysts featured in Prospect, LBC, Christian Science Monitor, Politico and more.

Over the past 12 days, the UK has been rocked by a shocking wave of far-right riots which have highlighted the dangerously symbiotic relationship between online extremism and real world violence. As those up and down the UK and around the world sought to make sense of the violence, ISD analysts have provided commentary to a broad range of media including the BBC’s Today programme on the growth of the UK far-right and BBC Newsnight on the radicalisation of young men; the New York Times on social media platforms’ failure to enforce their terms of service; and CNN on the role algorithms played in amplifying dangerous false claims.

Our rapid response research has focused on how online misinformation quickly translated into offline violence in the wake of the Southport stabbing; how far-right extremists targeted hate against Muslim and migrant communities; and how the decentralised online networks at the heart of this violence mobilised quickly and effectively. Our teams have used our analysis to engage with impacted communities, law enforcement, policymakers and regulators dealing with the fallout of the largest far-right extremist mobilisation in many years.

Because the violence seems to have largely subsided for now, we find ourselves at an inflection point where people are asking how we got here and what comes next. Over the coming weeks, ISD will focus on the longer-term policy implications including the prevention of future far-right violence, the need for a renewed cross-governmental counter extremism strategy, and the importance of strengthening regulatory responses for social media platforms. We look forward to engaging with partners around these important issues in the weeks ahead.

For now, this Insights summarises what we have learned about the riots and ISD’s analysis on the path forwards.

• From rumours to riots: How online misinformation fuelled violence
• ‘Total system collapse’: inciting violence on Telegram
• The foundations of violence: The growth of far-right hate in the UK
• A roadmap for counter-extremism in the UK
• Far-right seize on misinformation to incite violence
• Weekend riots spark fears of further far-right activity fuelled by misinformation
• What powers do the authorities have to hold platforms to account?
• Shining a light on the role of social media platforms in promoting hate

On Saturday, 13 July, Thomas Matthew Crooks attempted to assassinate former President Donald Trump at a campaign rally. The attack ended with an attendee and the shooter dead and Trump lightly wounded; it also catalysed the spread of myriad viral narratives promoting false claims across mainstream and fringe platforms. Within 24 hours, they had racked up more than 100 milion views on X alone.

Director of Technology and Society, ISD US, Isabelle Frances-Wright, Director of Threat Analysis and Prevention, ISD US, Katherine Keneally and Executive Director for Africa, the Middle East and Asia (AMEA) Moustafa Ayad investigated the murky world of unverified and often misleading information disseminated across social media. The shooter’s alleged identity morphed from an Italian sports commentator to a Chinese national to a troll whose images continue to be shared. In some cases, the accounts promoting the claims were self-described open-source intelligence (OSINT) analysts, who exploited uncertainty to drive false narratives.

ISD’s research also found that narratives mutated as they spread from fringe platforms such as Telegram and 4chan to more public spaces such as X (formerly known as Twitter), and vice versa.

Read the Dispatch here. Our findings were featured in The New York Times‘ reporting and also covered by France 24 and Raw Story.

Also in this newsletter:

• Terrorist audio content flourishes in TikTok sounds
• Protecting rights and democracy: the road ahead for the new UK government
• Our staff featured in ABC Australia, NBC News, Yahoo News and more.

On 24 May, a 20-year-old man livestreamed an attempted mass shooting at a grocery store in Indiana while espousing racial slurs. ISD’s Executive Director for Africa, the Middle East and Asia, Moustafa Ayad and Director of Threat Analysis and Prevention at ISD US Katherine Keneally, identified dozens of versions of the video across X (formerly Twitter) and Facebook. Together, they received almost 9 million views within 48 hours.

The spread of the video poses obvious questions about platforms’ ability to stop the spread of violent content, and points to potential gaps in the criteria set by cross-platform protocols designed to halt extreme material going viral. Though the attack itself thankfully did not lead to any physical injuries or fatalities, the potential psychological impact for viewers and those in the grocery store at the time remains significant.

In an interview with the Washington Post about the attack, Moustafa Ayad warned that one of the novel factors was the apparent “gamification” of the violence. For example, at one point the shooter asked viewers for input on who to target. “While there is no ideological link here, that was an aspect of it that was relatively novel and shocking,” said Moustafa.

Also in this newsletter:

• Online pro-Chinese Communist Party networks target Canada
• Right-wing extremists in Quebec push narratives about Israel-Hamas conflict
• Conspiracy theories continue to mobilise extremists to violence
• Israel-Hamas conflict galvanises the reanimated Canadian Jewish Defence League
• Our staff featured in Rolling Stone, the Tampa Bay Times, Sky News and more.

Last week, ISD released its Pulling Back the Curtain series, a four-part investigation digging into YouTube’s recommendation algorithm. Analysts found that YouTube is failing to properly safeguard young users: accounts set up to represent teens were served sexualised, misogynistic and suicide-related content.

The research, by Senior Analyst Aoife Gallagher, Digital Research Analysts Lucy Cooper and Rhea Bhatnagar, and Senior Digital Research Manager Cooper Gatewood, also discovered anti-vaccine content and clips of Andrew Tate, the misogynist ‘lifestyle guru’ banned by YouTube, in the platform’s search results and recommended videos.

The executive summary of Pulling Back the Curtain is available here. You can also read the in-depth investigations on the recommendations fed to accounts who showed interest in gaming, male lifestyle gurus, ‘mommy vloggers’ and Spanish language news.

Also in this newsletter:

• Russian influence operation Doppelganger linked to fringe advertising company
• Canada’s right-wing extremists and the Israel-Hamas conflict
• Antisemitism and anti-Muslim hate on Canadian YouTube after 7 October

• Network of Facebook pages and groups, ‘Patriots Run Project’, exposes Meta’s transparency failures
• Our staff features in WIRED, the Beg to Differ podcast, and more.

Following the guilty verdict against former US President Donald Trump in his New York hush money trial, many news reports focussed on the violent rhetoric emanating from spaces frequented by extremists.

ISD experts argue that singling out this rhetoric obscures the greater threat posed by the mainstreaming of anti-government and related conspiracy theories surrounding the verdict. ISD’s Jared Holt also highlighted to the Atlantic that some of the most extreme narratives are coming from those in positions of real power. These conspiracy theories – which have proven dangerous over and over again – have the potential to radicalise and influence individuals who are not fully invested in conspiracy theories but are open to being persuaded.

The manipulation of the guilty verdict, whether by powerful actors or extremist communities, will reverberate through the election campaign. It holds the potential to catalyse individual acts of violence while amplifying broader distrust in out democratic institutions.

Additional coverage in CNN.

Also in this newsletter:

• Pro-Kremlin campaigns intensify in Germany ahead of European elections
• How Meta’s news ban is disrupting discourse about the Israel–Hamas war
• Political violence, intimidation and threats mars Irish and EU election campaigns
• The Houthi (Ansar Allah) Digital Ecosystem
• The Russian propaganda nesting doll: How Russian state media is circumventing restrictions
• ISD’s CEO, Sasha Havlicek, joined the Viva Technology Conference in Paris.
• Our staff featured in WIRED, USA Today, Zeit Online and more.

Earlier in May, ISD and the Center for American Progress sent a letter to Meta executives on behalf of 51 organisations and researchers about the company’s decision to shut down the research tool CrowdTangle and replace it with the Meta Content Library after 14 August 2024, possibly disrupting US election oversight before and after November.

The letter outlines numerous concerns over the limitations of the tool’s replacement and makes two requests: 1) ensure the continued operation of CrowdTangle beyond Inauguration Day, through at least 31 January 2025; and 2) guarantee ongoing access to the Meta Content Library for organisations currently using CrowdTangle. 

“The gaps in functionality and access between CrowdTangle and the announced Meta Content Library, coupled with a lack of access for key partners, will lead to substantial vulnerabilities during a key election year,” the letter reads. “This decision jeopardises essential pre- and post-election oversight mechanisms and undermines Meta’s transparency efforts during this critical period, and at a time when social trust and digital democracy are alarmingly fragile.”

The letter was featured by Axios in its Tech Policy newsletter.

The full sign-on letter is available here.

Also in this newsletter:

• The US Senate’s AI policy roadmap is not a road to a safe future
• Ten years later, reverberations from the Bundy standoff continue to shape the anti-government landscape
• Pro-Kremlin propaganda about neo-Nazis in Ukraine targets South Africans online
• ISD staff featured in AP News, RTÉ, Carbon Brief, the Sydney Morning Herald and more.

ISD, along with other expert groups, contributed to the recently distributed guide to conspiracy theories in the UK for MPs and election candidates commissioned by Penny Mordaunt, the leader of the House of Commons.

“These (conspiracy) campaigns are also a threat to the health of our democracy. It is essential that we give the public and their representatives the tools they need to combat this phenomenon,” said Mordaunt. Her Labour counterpart Lucy Powell was quoted by the Evening Standard saying the guide was a “must-read for MPs and candidates.”

The guide highlights eight common conspiracies circulating in the UK, including QAnon, anti-vaccine narratives, pro-Russia narratives regarding the war in Ukraine, the Great Replacement, 5G, and misinformation about 15-minute cities.

Other contributing organisations included the Antisemitism Policy Trust, Full Fact, Tell Mama, the Community Security Trust, the Global Network on Extremism and Technology and the political analyst Arieh Kovler.

Coverage of the guide is also featured in The Guardian, Jewish News and Metro.

Also in this newsletter:

• Conspiracy theories matter, but not all are impactful: A guide for analysing risks to audiences
• UK Terrorgram proscription: Useful, but limited tool to combat online network
• Event: Jennie King, guest lecturer for the Stockholm Series speaking on the fight for information integrity, climate action & democracy
• ISD staff featured in El Mundo, the Boston Globe, Wired, the New York Times and more

The UK faces a fundamentally different threat from extremism and weaponised hate today than it did a decade ago. Both the far right and Islamists have been galvanised to action by recent events, particularly Hamas’ 7 October attack on Israel. In March, the government also unveiled a new definition of extremism, in a bid to help counter threats to the UK.

A new policy brief by ISD’s Director of Counter Extremism Policy and Research, Milo Comerford, and Hate and Extremism Analyst Hannah Rose explores the increasingly diverse range of threats to the UK’s security. It also provides a number of recommendations on how the government can work to combat these challenges.

Also in this newsletter:

• Hindu nationalists push conspiracy theories and misinformation ahead of elections
• Changing tides: Discourse towards migrants and asylum seekers on Facebook and X in Germany in 2023
• Workshop: Uniting against antisemitism; empowering transatlantic civil society responses across Latin America
• ISD contributes to shaping the European Commission’s DG-CLIMA policy regarding climate mis- and disinformation
• Our staff featured in Climate 1.5 Podcast, Politico Tech podcast, Irish Examiner and more.

Misinformation surged following Iran’s drone and missile attack on Israel on Saturday — a response to an unclaimed Israeli strike on an Iranian diplomatic mission in Syria on 1 April. X (formerly known as Twitter) in particular was awash with unverified and falsified images. This content, shared by ‘checkmark’ paid premium subscribers, self-proclaimed ‘OSINT’ (open-source intelligence) accounts, citizen journalists and even the Iranian government, received millions of views.

ISD’s rapid response analysis, led by our Executive Director for Asia, the Middle East and Africa, Moustafa Ayad, and Director of Technology and Society, Isabelle Frances-Wright, was covered in WIRED, Sky News, NPR and AFP. They found that within just seven hours of drones being launched, 34 images and videos claiming to show the ongoing conflict but which were false, misleading or generated using artificial intelligence (AI) had received over 37 million views on X.

Read more about it here.

Also in this newsletter:

• Pro-CCP Spamouflage campaign experiments with new tactics targeting the US
• Shared Endeavour Fund projects engage over 31,000 Londoners, particularly young people, in activities designed to address intolerance, hate and extremism
• Explainer on US ‘Antifa’ groups
• Event: Jiore Craig on disinformation & AI: Takeaways from Taiwan’s elections and moving forward in 2024
• Event: Stockholm Series: Jennie King on the fight for information integrity, climate action & democracy
• Our staff featured in the Hill, AFP, iNews, Tagesschau Faktenfinder, and more.