Misogynistic Pathways to Radicalisation: Recommended Measures for Platforms to Assess and Mitigate Online Gender-Based Violence

Authors: Sara Bundtzen
Published: 14 September 2023

This report is in English, but is also available in German

This policy paper was produced as part of the project Digital Policy Lab (DPL), funded by the German Federal Foreign Office. As part of the DPL, ISD organised working group meetings on the topic of online gender-based violence between May and June 2023. The working group comprised DPL members representing national ministries and regulators from Australia, Canada, Germany, New Zealand, the UK and the US. Members of the Global Partnership for Action on Gender-Based Online Harassment and Abuse (Global Partnership) and the Christchurch Call’s multi-stakeholder community joined the working group and contributed to this paper. Participants also included representatives from civil society, academia and industry.

While participants contributed to this publication, the views expressed in this paper do not necessarily reflect the views of all participants or any governments involved in this project.

This paper reviews online gender-based violence (OGBV) as existing within a continuum of (on- and offline) violence, emphasising the connections with different extremist ideologies, including the dissemination of terrorist and violent extremist content (TVEC). It aims to prioritise a gender perspective in responding to TVEC so that social media platforms can better intervene in and mitigate misogynistic pathways to radicalisation that can begin (or be reinforced) online. The discussion recognises that the mitigation of OGBV and online pathways to radicalisation requires a whole-of-society and whole-of-government approach. Whilst there are steps that governments and civil society can and should take, such as overseeing and enforcing emerging regulatory frameworks and voluntary commitments, this paper and its recommendations emphasise the role and actions of platforms.

Outlining the impact of OGBV at micro (individual) and macro (societal) levels, the paper considers the role platforms can play in exacerbating the risks of OGBV, evaluating platform policies, content moderation practices, user interface design and algorithmic recommender systems. In this context, the paper asserts that researching and mitigating the risks of OGBV can enable earlier warning of and intervention in misogynistic pathways to different forms of violent extremism. Reiterating that any mitigation of risks must come in support of users’ fundamental rights, including their right to privacy and freedom of expression, the paper proposes and elaborates on the following key recommendations:

  • Enable API access to publicly available data for public interest research;
  • Develop gender-disaggregated and standardised transparency reporting;
  • Apply a victim-survivor-centred Safety and Privacy by Design approach;
  • Enhance cross-platform cooperation and information sharing of OGBV incidents (including actors and tactics);
  • Review content moderation policies, processes, and systems to acknowledge the continuum of violence and misogyny as a vector for violent extremism;
  • Apply intersectional feminist knowledge in risk assessments of AI-based systems;
  • Strengthen and encourage multi-stakeholder dialogue and collaboration.
Download Report

Pulling Back the Curtain: An Exploration of YouTube’s Recommendation Algorithm

This is the executive summary of ISD’s Pulling Back the Curtain series which explores different aspects of YouTube’s recommendation algorithm. The series consists of four separate investigations examining the recommendations sent to eight U.S.-based YouTube accounts registered with varying ages, genders, and interests over one month. This report summarizes the findings across all four investigations and provides an overview of the project and platform policy recommendations.