X's Community Notes were 97% accurate regarding COVID-19 vaccines. What are the advantages and challenges of Community Notes?
Meta CEO Mark Zuckerberg announced in January 2025 that he plans to incorporate
Facebook Without Fact-Checkers Will Put Truth to the Test - Bloomberg
https://www.bloomberg.com/news/articles/2025-01-17/facebook-without-fact-checkers-will-put-truth-to-the-test
In recent years, fact-checking interventions on social media have accelerated and intensified due to the threat of COVID-19. Of course, many of the posts deleted by fact-checking were misinformation, such as unfounded conspiracy theories and unscientific and dangerous claims. However, there were also cases where evidence-based and highly accurate information was mistakenly checked.
On August 4, 2021, Instagram fact checkers removed a post about ivermectin. The post was from Cochrane , a nonprofit organization that collects and summarizes medical research, which said, 'After a thorough review of the available data, we found no evidence to support the efficacy of ivermectin as a treatment for COVID-19.'
Cochran appealed Instagram's decision, but about a year after the deletion, he received a notice that the post violated the community guidelines. During this time, Cochran's account was shadowbanned, so he had not received any mentions from other users.
On the other hand, X (formerly Twitter) uses a 'community note' system where fact-checking is done by the user community rather than the social media platform.
'Community Notes' is characterized by the fact that some users write notes on posts, which other users then rate as useful or not. Furthermore, in this system, users rate notes as 'useful' or 'unuseful', and if their rating matches the final overall judgment, they gain points. Conversely, if a note rated 'unuseful' is ultimately deemed 'useful', they lose one point, and if a note rated 'useful' is ultimately deemed 'useless', they lose two points.
In fact, a 2024 study found that experts randomly selected and evaluated 205 community notes about COVID-19 vaccines and found that 97% of them were accurate. In addition, about half of them cited high-quality sources such as scientific papers.
Professor Kucharski points out that a 'smart crowd' is necessary for the community note function to work, and that this requires 'independence of evaluations that are not influenced by others' and 'diversity of participants' perspectives and knowledge.' He goes on to say that X's community note function basically meets these conditions, and that the need for consensus from a diverse range of users reduces systematic bias.
However, while Community Notes essentially values independent evaluation and diverse viewpoints, it can be influenced by other posts on the same topic, and even users who normally have diverse opinions may be biased in their evaluation of a particular topic, Professor Kucharski points out.
In addition, Professor Kucharski said that even if a post containing incorrect information was noted within a few hours of being discovered, by that point interest in the post had already waned among users overall, and in many cases the post did not receive enough positive feedback from users. As a result, there is also the issue that the post is left uncorrected.
Furthermore, Professor Kucharski questioned whether the 97% accuracy rate of the COVID-19 vaccine community notes was high enough, given that millions of users could be exposed to dangerous information. In addition, he pointed out that dealing with content whose accuracy is difficult to determine is also a challenge.
While Professor Kucharski acknowledged the potential of community notes, he pointed out that there are several challenges, including timing, quality of evaluation, standards of accuracy, and dealing with complex content.
Related Posts:
in Web Service, Posted by log1i_yk