Social media platforms have a transformative impact on how people connect and communicate, how they express, search for and encounter information, and how they organize themselves. Many changes have been positive, but the enabling dynamics of these platforms has also opened up the possibility of being exploited by organized hate groups to organize attacks or to intimidate and harass members of minorities. Historical discrimination against various ethnic minorities has found new channels and is thriving online. As early as 2003, the OSCE Ministerial Council Decision on Tolerance and Non-Discrimination recognized that racist and other hateful content on the Internet can incite hate crimes. The persistence of these problems was highlighted in a recent statement by the UN Special Rapporteur on Minority Issues when he accused social media of hate speech as a direct contributor to the rise in hate crimes against minorities and called for recognition and approval of this “intoxication” on the internet confront.
Online hate content is not an anomaly, but a persistent feature of the internet that disproportionately affects vulnerable people. With an estimated 10 to 12 million Roma living in Europe, these communities are the largest ethnic minority on the continent. At the same time, 80% of Roma men and women live at risk of poverty. Their low economic and social status persists against a background of persistent discrimination – they are the most unpopular and most discriminated minority in Europe, with more than half of citizens having negative attitudes towards Roma. Such attitudes are sometimes artificially and organized reinforced in online rooms. There has been an increasing wave of anti-Roma rhetoric in recent years, and the coronavirus pandemic has exacerbated this trend. Despite the urgency of these issues, the impact of hate speech directed against minorities and marginalized groups on social media has not been fully recognized by providers.
Hate speech turns into hate crimes
Internet hatred plays an important role in normalizing racist and xenophobic attitudes. Posts and comments that portray negative stereotypes and prejudices can increase interethnic tensions. They can also lead to a negative spiral when manifestations of hatred encourage more hatred or even escalate into physical harm.
Racist incidents usually develop according to the following paradigm: A right-wing group publishes a xenophobic article about the Roma. This post provokes a growing level of hostility on the internet. For example, others express their support for these views in the comments, the post becomes known, even viral, and then there is an attack. This information is not verified and is often created specifically to justify the attack. In 2018 there were attacks on temporary Roma settlements in Ukraine, which were openly preceded by anti-Roma posts inciting violence. These and similar cases show that decisions about online content play an important role in shaping events with consequences for life and death.
The pandemic has sparked a new wave of anti-Roma sentiment and hate speech, and fueled violence in several European countries where Roma are persistently held responsible for violating quarantine restrictions and for spreading or causing COVID-19. One of the defining characteristics of such posts is that they provoke a violent reaction from other users who post hateful comments inciting hatred, violence, displacement, bullying, and ongoing social stigma. Another trend is the use of pejorative metaphors when referring to Roma for the precise purpose of dehumanizing the Roma as a group. This is often done in online groups that can be specifically created to spread hatred against Roma.
Comments referring to the incompetence of the police in maintaining law and order and preventing investigations into alleged anti-social behavior or crimes are particularly dangerous. Such reactions provoke people to “take matters into their own hands”, claiming that they are the only ones who can restore social order. In this way, the hate groups delegitimize state institutions and create a justification for violent acts.
Content moderation practices
Large digital service providers have taken steps towards more responsible content moderation. However, the efforts are mostly reactionary – after neglect of online hate speech has had dire real-life consequences. Vendors are also often too US-focused, and marginalized communities are not given adequate attention. However, given the growing opportunities to spread hate speech on these platforms, more active and targeted efforts are needed to prevent future harm.
Efficient content moderation is not an easy task. It is left to the vendors to calibrate the red lines of what should be considered as remaining and pending content, and the risk of overregulation is great. Freedom of expression can be restricted by law in cases where it incites violence. In such cases, there must be a close connection between the statement in question and a significant risk of harm. Despite many cases that offer such a context, hateful content remains present on social media platforms.
The providers are contradicting their existing standards, which have long outlawed bullying, hate speech and incitement to hatred and violence. They also often lack the human content moderators who would understand the context of such situations. Additionally, it is important to remember that while such binary decisions are a major human rights issue, they are not the only way to reduce security risks. Social media algorithms can also regulate the entire flow of posts. The decisions about which platforms add or increase friction are the most important ones when moderating content and the most important ones in high risk situations.
As Facebook has shown in the past in emergency situations, it can effectively limit the spread of malicious content that its systems predict is likely to violate the company’s community standards. The company can reduce toxic levels by turning the spread of flammable items down, thereby limiting spread and exposure. This method prevents borderline content and makes the platform a safer place. These methods are used temporarily at high-risk locations. Recent examples are risks related to civil unrest and uprisings in Myanmar, Ethiopia and Sri Lanka. The security risk is then not reduced by changing the rules on hate speech, but by reducing the virality of problematic content in personalized newsfeeds. This method addresses the main difference between online and offline hate speech – the ability to spread it quickly – when hate statements are shared, encouraged, repeated and included in the comment section, resulting in injury and crime in real life.
Further automation and content recognition technologies can be particularly helpful in the case of ethnic tensions with a high potential for escalation into a conflict. The problem remains that content is often highly context-dependent, which means that automation can fail or generate false positive results and lead to a direct or indirect violation of users’ rights to freedom of expression. The automation is still insufficient to understand and interpret the context associated with the post, the user’s intention, and the linguistic and sociological context of the post. The evaluation of hate speech is highly context-dependent. This poses a persistent problem for global platforms looking for scalable solutions. Hate speech is often not clear; it tries to play with words, forms and rules in order to escape the formalized language. On the other hand, if a qualified human check is not guaranteed, it can restrict legitimate bodies and lead to greater collateral damage, especially since the rules of enforcement are intransparent.
These approaches are unlikely to work in isolation and must be designed to work together. In addition, the responsibility for preventing hate crimes also rests on states. While hate speech laws and norms vary around the world, content inciting civil unrest or criminal offense may be considered a deliberate act of incitement to racial hatred and is illegal in many countries. What is missing, however, are precise legal definitions and mechanisms to effectively monitor their implementation, which, coupled with victims’ low level of trust in the police, leads to a high level of under-reporting of such cases. What is also missing is specialization and local capacities in the criminal justice systems – with law enforcement officers and public prosecutors. Authorities should invest in building their capacity and confidence to protect individuals and communities from harm, including by drawing on the experiences of victims and civil society organizations who have documented hate online and translated it into real-life incidents.
In addition to the challenges of the legislation and its applicability, there are also practical difficulties in removing hateful material. Once the content is posted on the internet, it is shared across multiple accounts and platforms, which makes it problematic to completely remove all of the content. Cooperation between companies and governments remains limited for the time being, split over questions about effective approaches to moderating content and the corresponding regulation.
The way forward
Current methods of content moderation are unsustainable. The standards need to be viewed in terms of the communities they affect, especially in cases of marginalized and vulnerable communities in high risk environments. Both platforms and states should ensure a meaningful participation of the target persons and their perspectives in the development and advice on the creation of standards and their implementation.
To support this process, we need more transparent processes and more efficient data collection. Disaggregated data is needed to understand the extent and impact of persistent hate speech on communities. The platforms that exert global influence must begin with moderation in the local context. They need to take proactive action, adding much-needed context and resources to the way they operate, monitor, and enforce their standards. Appropriate moderation of content must enable a more security-sensitive and risk-averse online environment.
Social media content that causes offline harm is not limited to Roma in Europe. It’s a global problem. While the people who suffer most from hate speech are the ones who are attacked, hate speech divides societies and increases the general level of aggression and insecurity for all. Less hate speech online means more security and more resilient societies. Platforms shouldn’t wait for the damage to be done to respond.
Further reading on e-international relations