YouTube is Profiting from Misinformation and Calls for Violence via its Super Chats Tool

12 May 2022

By Ciarán O’Connor

YouTube’s Super Chats tool was launched in 2017 to allow creators to monetize their livestreams and enable viewers to pay to pin comments on live video chat feeds. But since their launch, Super Chats have also become a source of revenue for numerous political coverage and commentary channels, conspiracy theorists and even extremists on the platform. 

ISD research has found that Super Chats are allowing both creators and YouTube itself to profit from live chats promoting violence, conspiracies, misinformation and hate. 

This Dispatch, based on a larger report recently released by ISD, covers key findings from this research. It then examines one of the channels analyzed in closer detail, Timcast IRL, to highlight how the tool is used and misused by creators on the platform. It illustrates YouTube’s failure to effectively enforce its own community guidelines on Super Chats.
_________________________________________________________________________________

Key Findings

Super Chat comments are used to publish content that violates YouTube’s Community Guidelines. Across all three case studies featured in the research, ISD found that 5% of a sample of paid Super Chat comments contained content that promoted conspiracies, mis- and disinformation, hate speech or threats of violence.

YouTube’s policies are not matched by enforcement of content moderation of live chat comments. ISD’s research illustrates that YouTube’s Community Guidelines are not being enforced consistently or effectively when it comes to live chat content like Super Chats.

Popular YouTube creators are not required to use moderators on live chat feeds. YouTube recommends, but does not require, that creators assign moderators in their live chats, especially for creators who host “high-traffic events”. YouTube’s decision against requiring in-chat moderators represents a significant failing in safeguarding users.

Super Chats offer creators a significant source of revenue. ISD recorded over $219,000 in paid Super Chat comments posted on just one YouTube channel included in the research.

January 6 catalyzed calls to violence and harassment for numerous YouTube users. ISD observed many paid Super Chat comments posted on live videos related to the protests in Washington DC on January 6, before, during and after the violent events of the day. At the time of writing, these comments are still live on YouTube.

YouTube offers no effective data access to research live chat content on its platform. YouTube offers no analytical tools (such as an API) to track or analyze live chat content at scale across the platform, meaning such content is outside the scope of most analysis or research.

Table 1: Summary of the key findings from each YouTube channel analyzed, including number of videos, total Super Chat comments, revenue and revenue split between the YouTube channel and platform (YouTube).

Table 2: Summary of key findings of content promoting mis/disinformation, conspiracy theories or threats of violence from each YouTube channel analyzed, including number of videos, revenue and revenue split between the YouTube channel and platform (YouTube).

Case Study – Timcast IRL

ISD analyzed 100 videos broadcast between October 2020 – November 2021 on the Timcast IRL channel, run by podcast host and political commentator Tim Pool. ISD also recorded every Super Chat payment and comment posted on these videos. In total, 13,839 Super Chats were published during these video broadcasts, amounting to $219,416.22 that was sent to the Timcast IRL channel. Bearing in mind the 70/30 revenue split between channel and platform for Super Chat payments, these comments earned Timcast IRL $153,591.35 and YouTube $65,824.86

ISD analyzed the sample of 13,839 Super Chat comments for terms promoting misinformation, false claims, conspiracy theories and/or threats of violence and identified 720 comments that fit this criteria, accounting for 5% of the total sample. These 720 comments equate to $11,647.87 (also 5% of the total amount of money spent on the channel). Taking into account the 70/30 revenue split, these comments earned Timcast IRL $8,153.50 and YouTube.

Analysis

2020 Presidential Election

ISD recorded 188 paid Super Chat comments on Timcast IRL videos that referenced false or misleading claims about mail-in ballots, supposed ballot stuffing, voter fraud and conspiracy theories about Dominion Voting Systems, all of which violate YouTube’s guidelines regarding election misinformation.
13 Super Chat comments posted in these videos directly referenced claims about mail-in ballots. One of these comments stated, “Democrats are gonna forge mail in ballots to win again in 2024” ($20 Super Chat (SC)). Another claimed that 1.8 million mail-in ballots were sent to voters in Pennsylvania but 2.5 million votes were counted, alleging this was proof of fraud ($4.99 SC). The same claim was the subject of an Associated Press fact check when it went viral in December 2020.

Fig 1: $4.99 Super Chat comment posted on Timcast IRL. YouTube earned $1.50 from this comment.

23 Super Chat comments directly referenced Dominion Voting Systems. In one example, a YouTube user posted the text of an article from the conspiracy website

Natural News which claimed to show Dominion voting machines being hacked ($20 SC). Another comment claimed that data uncovered on Hunter Biden’s laptop confirmed Dominion had ties to a “Chinese banking firm” ($17.38 SC). The same claim was the subject of a fact check by USA Today.

January 6 and Threats of Violence

In the Timcast IRL video broadcast on January 5, 2021, ISD identified four messages advocating the use of violence at the Capitol the following day. These included the messages:

• “Prepare yourselves – the wicked and immoral will be annihilated – this is the only way” ($6.66 SC);
• “The only effective threat is the threat to become ungovernable. An orderly protest has no power” ($5 SC); 
• “The time of war is here. Only a fool still has faith in the courts” ($4.99 SC); 
• “…I have no problem leaving Antifa weaklings bleeding out in the street…” ($20 SC).

A sample of other pro-violence Super Chat comments posted on post-January 6 broadcasts include:

• “For those saying it did nothing, the most powerful people in the world were laying on the ground and hiding behind chairs because, for a moment, their bubble that made them untouchable was popped.” ($20 SC)
• “Violence works… violence towards the establishment works EXTREMELY well” ($20 SC)
• “Only use violence if necessary!” ($19.99 SC)
• “Violence may not produce results but the threat of it or fear of [sic] can.” ($10 SC)
• “I was there!! Pence tweeted he wasn’t going to take a stand. some yelled let’s take the building. we made a point. we will not be silenced anymore.” ($20 SC)

Fig 2: $20 Super Chat comment posted on Timcast IRL. YouTube earned $6 from this comment.

COVID-19 Conspiracy Content

ISD identified 30 Super Chat comments posted during Timcast IRL broadcasts that contained false or misleading content in reference to vaccines, which directly violated the platform’s guidelines related to COVID-19 misinformation. For example, one comment stated, “The vaccine is poison Tim. You will come to see it in time” ($10 SC), while another claimed 100 people died during a trial of the “[Bill] Gates vaccine… within 7 days” of receiving the vaccine. ($4.99 SC). This claim was the subject of a Reuters fact check.

Fig 3: $10 Super Chat comment posted on Timcast IRL. YouTube earned $3 from this comment.

At least three comments included false claims about substances found in the vaccine ingredient, violating YouTube’s COVID-19 misinformation guidelines. Three others propagated claims that the vaccines are a cover up, part of a sinister plot by world elites to enact population controls across the globe. This included one comment which described vaccines against a “lab made virus” as akin to “genocide.” ($5 SC)

Chauvin, Rittenhouse Trials, BLM Commentary and Potential Hate Speech Violations

While examining Super Chats comments posted on Timcast IRL broadcasts that covered the trials of Derek Chauvin and Kyle Rittenhouse, ISD identified various comments referencing both men, as well as George Floyd, Jacob Blake and the wider Black Lives Matter (BLM) movement, that appear to violate YouTube’s policies against hate speech.

ISD identified other comments that were closely tied to wider racist comments aimed at Black people. Examples of this include the comments: “My fear is, the BLM narrative that rewards the families of black victims killed in police shootings (justified or not) with settlements and GoFundMe’s will create the same type of system Terrorist networks use to recruit poor suicide bombers” ($50 SC), and “BLM and GoFundMe accounts [are] the new AlQuaeda. If I’m poor, black, POC, and I know suicide by cop can net me millions as the current trends suggest.” ($200 SC)

Fig 4: $200 Super Chat comment posted on Timcast IRL. YouTube earned $60 from this comment.

Conclusion

Super Chats represent an under-researched element of content monetization and content moderation on YouTube. There are no analytical tools offered by YouTube, such as API access, for researchers making any scalable analysis beyond the scope of most research projects.

The findings of this analysis also highlight how there is a particularly stark enforcement gap on YouTube when it comes to Super Chats. While YouTube has developed nuanced policies designed to prohibit harmful or misleading content, the platform is failing to implement these policies across the Super Chats feature.

ISD recommends that YouTube and Google address this gap, in part by requiring, rather than just recommending, that creators who broadcast “high-traffic events” use moderators in live chats. This shift would go some way to ensuring that explicitly violative content is not permitted to act as a source of monetization for creators. 

Read the full report, including expanded analysis and notes about methodology, here.

Ciarán O’Connor is an Analyst at ISD, working in the Research and Policy unit.