Authors: Sara Bundtzen
Published: 9 December 2022
This Policy Paper was produced as part of the project Digital Policy Lab (DPL), funded by the German Federal Foreign Office. The responsibility for the content lies exclusively with ISD Germany. The DPL is an inter-governmental working group focused on charting the policy path forward to prevent and counter the spread of disinformation, hate speech, extremist and terrorist content online. It is comprised of representatives of relevant ministries and regulatory bodies from liberal democracies.
With the volume of content increasing and user attention spans remaining fixed, most online platforms moved on from reverse-chronological feeds to algorithmic ranking to show users the “most interesting” content – with “most interesting” usually being content that has the highest predicted value to a company, for example, increasing the average time users spend on a platform.
Algorithmic ranking systems make automated decisions about which pieces of content to prioritise or demote on feeds or in search results, who to connect with, who or what pages to follow – ultimately shaping the online experience of billions of users.
This paper unravels the use of algorithmic ranking and its role in shaping online discourses and behaviour. It looks at the infamous “engagement problem” and other algorithmic phenomena, and how they may exacerbate the spread of harmful or “borderline” content, while reinforcing biases and discrimination. Acknowledging a knowledge gap and lack of clear evidence when it comes to the aggregate effects of algorithmic ranking on society, this paper examines methodological and epistemic challenges of algorithmic auditing and recognises the need for common quality standards.
It further examines the benefits and drawbacks of proposed interventions around increased user agency, middleware, positive friction and nudges, “quality-focused” as well as “bridging-based” ranking practices. Throughout, the paper considers case studies of industry practices as well as regulatory (or co-regulatory) initiatives proposed by liberal democratic governments.
Sara Bundtzen is an Analyst at ISD Germany, where she studies the spread of information manipulation by state and non-state political actors in multilingual online environments. As part of the Digital Policy Lab (DPL) team, Sara informs ISD’s advisory work and analyses proposed pathways toward countering disinformation, influence campaigns, hate speech, and extremist content.
This publication is also available in German.