Welcome to VOX-Pol’s Online Library, a research and teaching resource, which collects in one place a large volume of publications related to various aspects of violent online political extremism.

Our searchable database contains material in a variety of different formats including downloadable PDFs, videos, and audio files comprising e-books, book chapters, journal articles, research reports, policy documents and reports, and theses.

All open access material collected in the Library is easy to download. Where the publications are only accessible through subscription, the Library will take you to the publisher’s page from where you can access the material.

We will continue to add more material as it becomes available with the aim of making it the most comprehensive online Library in this field.

If you have any material you think belongs in the Library—whether your own or another authors—please contact us at and we will consider adding it to the Library. It is also our aim to make the Library a truly inclusive multilingual facility and we thus welcome contributions in all languages.


Full Listing

Proposals for Improved Regulation of Harmful Online Content
2020 Benesch, S. Report
This paper offers a set of specific proposals for better describing harmful content online and for reducing the damage it causes, while protecting freedom of expression. The ideas are mainly meant for OSPs since they regulate the vast majority of online content; taken together they operate the largest system of censorship the world has ever known, controlling more human communication than any government. Governments, for their part, have tried to berate or force the companies into changing their policies, with limited and often repressive results. For these reasons, this paper focuses on what OSPs should do to diminish harmful content online. The proposals focus on the rules that form the basis of each regulation system, as well as on other crucial steps in the regulatory process, such as communicating rules to platform users, giving multiple stakeholders a role in regulation, and enforcement of the rules.
Angry by design: toxic communication and technical architectures
2020 Munn, L. Article
Hate speech and toxic communication online is on the rise. Responses to this issue tend to offer technical (automated) or non-technical (human content moderation) solutions, or see hate speech as a natural product of hateful people. In contrast, this article begins by recognizing platforms as designed environments that support particular practices while discouraging others. In what ways might these design architectures be contributing to polarizing, impulsive, or antagonistic behaviors? Two platforms are examined: Facebook and YouTube. Based on engagement, Facebook’s Feed drives views but also privileges incendiary content, setting up a stimulus–response loop that promotes outrage expression. YouTube’s recommendation system is a key interface for content consumption, yet this same design has been criticized for leading users towards more extreme content. Across both platforms, design is central and influential, proving to be a productive lens for understanding toxic communication.
Repeated and Extensive Exposure to Online Terrorist Content: Counter-Terrorism Internet Referral Unit Perceived Stresses and Strategies
2020 Reeve, Z. Article
U.K. Metropolitan Police Counter-Terrorism Internet Referral Unit (CTIRU) Case Officers (COs) are tasked with identifying, and facilitating the removal of material that breaches the Terrorism Act 2006. COs are extensively and repeatedly exposed to material deemed illegal and harmful (including but not restricted to graphic terrorist and non-terrorist material). However, there is little research on the impact of this work, or how COs manage and mitigate the risks of their roles. Semi-structured interviews reveal the adaptive coping mechanisms that promote good perceived health and wellbeing in CTIRU, as well as areas of concern and improvement.
Far-Right Extremism: Is it Legitimate Freedom of Expression, Hate Crime, or Terrorism?
2020 Lowe, D Article
Following the rise in far-right inspired terrorist attacks globally, social media and electronic communications companies have been criticized, mainly by politicians, for allowing far-right extremist content to be available. This article is a comparative legal study focusing on Australia, Canada, New Zealand, the U.K., and the U.S.’ legal provisions regarding the right to freedom of expression, hate crime, and proscription of terrorist organizations. This study found a disparity in the form of expression protected under this right. This disparity widens further when related to hate crime and proscribing groups as terrorist organizations. As such, social media and communications companies have difficulty setting at global level a baseline in determining whether content is legitimate commentary or is extremism promoting or inciting hatred and violence. The article concludes with a recommendation for how states can provide comparable legislation on hate crime as they have done in relation to Islamist inspired extremism. This will assist social media and communications companies in removing content and suspending accounts. These companies are not the guardians of freedom of expression, that is the role of states’ legislatures and judiciary.
The Propaganda Pipeline: The ISIS Fuouaris Upload Network on Facebook
2020 Ayad, M. Report
This new investigation from the Institute for Strategic Dialogue (ISD) delves into the inner workings of a pro-ISIS account network on Facebook, providing a case study of the resilient network dynamics, technological loopholes, and cross-platform activity that allowed a web of accounts to survive and flourish for over three months on a platform which purports to be a hostile environment for terrorist actors.
The Role of the Internet in Facilitating Violent Extremism: Insights from Former Right-Wing Extremists
2020 Gaudette, T., Scrivens, R. and Venkatesh, V. Article
While a growing body of evidence suggests that the Internet is a key facilitator of violent extremism, research in this area has rarely incorporated former extremists’ experiences with the Internet when they were involved in violent extremism. To address this gap, in-depth interviews were conducted with ten Canadian former right-wing extremists who were involved in violent racist skinhead groups, with interview questions provided by thirty Canadian law enforcement officials and ten community activists. Participants were asked about their use of the Internet and the connection between their on- and offline worlds during their involvement in the violent right-wing extremist movement. Overall, our study findings highlight the interplay between the Internet and violent extremism as well as the interactions between the on- and offline worlds of violent extremists. We conclude with a discussion of study limitations and avenues for future research.
Digitale Worte – Analoge Taten: Eine fallgestützte Analyse nach außen und nach innen kommunizierter Ideologie einer rechtsextremen Gruppierung
2020 Schwarz, K., Hartung, F., Piening, M.T., Bischof, S., Fernholz, T., Fielitz, M., Patz, J., Richter, C., Diskriminierung, F. and Hasskriminalität, F. Article
In diesem Beitrag werden auf Basis von staatsanwaltschaftlichen Verfahrensakten, Daten der Programmierschnittstelle von Twitter (API) und frei zugänglichen Online-Inhalten Konstitutions- und Kommunikationsdynamiken einer rechtsextremistischen Gruppierung analysiert, von der mehrere Mitglieder 2014 wegen der Bildung einer kriminellen Vereinigung verurteilt wurden. Es wird rekonstruiert und analysiert, wie die jungen Erwachsenen über verschiedene Kommunikationskanäle innerhalb ihrer Gruppe und nach außen kommunizieren, wie sich interne und externe Selbstdarstellungen unterscheiden. Dabei wird aufgezeigt, welchen Stellenwert Gewalt und Gewaltbefürwortung in dieser Gruppe besitzen, wie sich die Mitglieder als Individuen und als Kollektiv definieren, Geltung verschaffen und von anderen abgrenzen wollen. Da für die Kommunikation der Gruppe nicht zuletzt Social Media eine Rolle spielen, wird auch diskutiert, inwiefern Mechanismen digitaler, netzwerkbasierter Kommunikation, wie die sogenannte Filterblase, für den Radikalisierungsprozess relevant sind.
The homogeneity of right-wing populist and radical content in YouTube recommendations
2020 Röchert, D., Weitzel, M. and Ross, B. Article
The use of social media to disseminate extreme political content on the web, especially right-wing populist propaganda, is no longer a rarity in today's life. Recommendation systems of social platforms, which provide personalized filtering of content, can contribute to users forming homogeneous cocoons around themselves. This study investigates YouTube's recommendations system based on 1,663 German political videos in order to analyze the homogeneity of the related content. After examining two datasets (right-wing populist and politically neutral videos), each consisting of ten initial videos and their first and second level recommendations, we show that there is a high degree of homogeneity of right-wing populist and neutral political content in the recommendation network. These findings offer preliminary evidence on the role of YouTube recommendations in fueling the creation of ideologically like-minded information spaces.
Online Extremism: Challenges and Opportunities in the Western Balkans
2020 Comerford, M. and Dukic, S. Policy
The Western Balkans faces a double challenge from online extremism. Online platforms are facilitating the specific targeting of the region by diverse international extremist narratives. Meanwhile regional histories and geopolitics are being appropriated to justify extremist actions and narratives around the world. This is part of a wider trend that underscores the growing challenge posed by the proliferation of transnational extremist ideologies on online platforms, both violent jihadist and extreme right wing. In the Western Balkans, this poses a number of specific risks. While the issue of the prevention, mitigation, and regulation of online extremism is a global one, there are a number of region-specific considerations relevant to effective policy and practitioner responses.
Tiered Governance and Demonetization: The Shifting Terms of Labor and Compensation in the Platform Economy
2020 Caplan, R. and Gillespie, T. Article
Social media platforms have profoundly transformed cultural production, in part by restructuring the terms by which culture is distributed and paid for. In this article, we examine the YouTube Partner Program and the controversies around the “demonetization” of videos, to understand these arrangements and what happens when they shift beneath creators’ feet. We use the testimony of YouTubers, provided in their own videos, to understand how creators square the contradiction between YouTube’s increasingly cautious rules regarding “advertiser-friendly” content, its shifting financial and algorithmic incentive structure, and its stated values as an open platform of expression. We examine YouTube’s tiered governance strategy, in which different users are offered different sets of rules, different material resources, and different procedural protections when content is demonetized. And we examine how, especially when the details of that tiered governance are ambiguous or poorly conveyed, creators develop their own theories for why their content has been demonetized—which can provide some creators a tactical opportunity to advance politically motivated accusations of bias against the platform.
Jumanji Extremism? How Games and Gamification Could Facilitate Radicalization Processes
2020 Schlegel, L. Article
While the last years have seen increased engagement with gaming in relation to extremist attacks, its potential role in facilitating radicalization has received less attention than other factors. This article makes an exploratory contribution to the theoretical foundations of the study of gaming in radicalization research. It is argued that both top-down and bottom up gamification have already impacted extremist discourse and potentially radicalization processes but that research on gamification in other contexts points to a much wider application of gamification to extremist propaganda distribution tools in the future. The potential influence of video games on radicalization processes exceeds the transfer of the popular argument that exposure to violent media leads to desensitization to the context of radicalization and includes the exploitation of pop culture references, increases in self-efficacy regarding violence, and the direct experience of retropian visions through the content of games.
2020 Guhl, J. and Davey, J. Report
The briefing highlights how through its limited content moderation policies Telegram has become a safe space for white supremacists to share and discuss a range of explicit extremist material. Furthermore, it shows that through the many of these Telegram communities have become permissive environments where overt calls for violence and support for terrorism is widespread. Much of the content which we have identified appears to breach Telegram’s terms of service which prohibit the promotion of violence, suggesting that the platform’s current enforcement of its policies is not effective.
#Hamas: A Thematic Exploration of Hamas’s English-Language Twitter
2020 Margolin, D. Article
As the debate on whether Hamas should be designated a terrorist organization intensifies across Europe and North America, policymakers and practitioners seek to identify the core principles that unify the group and its ideology. This paper contributes to this discussion by examining how Hamas uses Twitter to frame its narrative to English-speakers around the world. From March 2015 until November 2019, when its account was suspended from Twitter, Hamas operated an English-language Twitter handle under the name @HamasInfoEn. Using thematic content analysis to explore the first 2,848 tweets sent by Hamas in English—between March 2015 and March 2018—this paper explores the socio-political and religious narratives that lay at the core of Hamas’s online public diplomacy throughout its first three years on Twitter. Since its entrance into politics in 2006, some academics argue that Hamas has increasingly sought to distance itself from acts of terrorism and legitimize its actions as a governing actor, thereby seeking to carve out a place for itself in the international community. This study presents a nuanced understanding of how Hamas represents itself internationally, to better understand where the group is going, and how to best counter its narratives.
Code of Conduct on Countering Illegal Hate Speech Online: Results of the 5th monitoring exercise
2020 European Commission Policy
To prevent and counter the spread of illegal hate speech online, in May 2016, the Commission agreed with Facebook, Microsoft, Twitter and YouTube a “Code of conduct on countering illegal hate speech online”. The implementation of the Code of Conduct is evaluated through a regular monitoring exercise set up in collaboration with a network of organisations located in the different EU countries. Using a commonly agreed methodology, these organisations test how the IT companies are implementing the commitments in the Code.
Combatting Terrorist Propaganda
2020 Ford, P. Article
Propaganda is a common element in many, if not all, terrorist incidents. Objectives are usually to instil fear in the target population but may also include winning approval from associates and inspiring new adherents. The advent of social media has greatly expanded opportunities for achieving these objectives. The attacks on worshipers in two mosques in Christchurch, New Zealand, on 15 March 2019 resulting in the murder of 51 people and the live streaming of the attack on Facebook were met with responses from governments in New Zealand, France and Australia that sought international cooperation in finding effective counter-measures to the propaganda value of the attacks. Australia’s ambitious regulatory approach is problematic when considered in an international context particularly in relation to the need to take account of free speech guarantees. However, since it recognises the need for international solutions in this area, there are opportunities to explore other ways of achieving its objectives.
An Online Environmental Scan of Right-wing Extremism in Canada
2020 Davey, J., Guerin, C. and and Hart, M. Report
This report represents the interim findings of a two-year study designed to increase understanding of the social media footprint of right-wing extremism (RWE) in Canada. This work is part of a larger project designed to understand RWE in Canada led by Ontario Tech University (OTU), in partnership with Michigan State University and the University of New Brunswick. This team are currently working on a similar project designed to map offline RWE in Canada. The project follows a similar study delivered in 2015, enabling researchers and policymakers to understand how RWE has changed in the past five years.
Antisemitism on Twitter: Collective Efficacy and the Role of Community Organisations in Challenging Online Hate Speech
2020 Ozalp, S., Williams, M.L., Burnap, P., Liu, H. and Mostafa, M. Article
In this article, we conduct a comprehensive study of online antagonistic content related to Jewish identity posted on Twitter between October 2015 and October 2016 by UK-based users. We trained a scalable supervised machine learning classifier to identify antisemitic content to reveal patterns of online antisemitism perpetration at the source. We built statistical models to analyze the inhibiting and enabling factors of the size (number of retweets) and survival (duration of retweets) of information flows in addition to the production of online antagonistic content. Despite observing high temporal variability, we found that only a small proportion (0.7%) of the content was antagonistic. We also found that antagonistic content was less likely to disseminate in size or survive for a longer period. Information flows from antisemitic agents on Twitter gained less traction, while information flows emanating from capable and willing counter-speech actors—that is, Jewish organizations—had a significantly higher size and survival rates. This study is the first to demonstrate that Sampson’s classic sociological concept of collective efficacy can be observed on social media (SM). Our findings suggest that when organizations aiming to counter harmful narratives become active on SM platforms, their messages propagate further and achieve greater longevity than antagonistic messages. On SM, counter-speech posted by credible, capable and willing actors can be an effective measure to prevent harmful narratives. Based on our findings, we underline the value of the work by community organizations in reducing the propagation of cyberhate and increasing trust in SM platforms.
Vom Analogen ins Digitale. Eine kurze Geschichte der dschihadistischen Propaganda und ihrer Verbreitung
2020 Zabel, M. Chapter
Der Beitrag bietet eine kurze Einführung in die Geschichte dschihadistischer Propaganda von den 1980er Jahren bis in die Gegenwart. Nicht nur die medientechnologische und -kulturelle Entwicklung, sondern auch geopolitische Konfliktgeschichte mit wichtigen Stationen wie der sowjetischen Intervention in Afghanistan und dem ersten Tschetschenienkrieg sind hier ausschlaggebend. Was die Entwicklung auf dem Feld der Online-Propaganda betrifft, stützt sich der Verfasser auch auf eigenen Erfahrungen im Rahmen der Beobachtung und Analyse dschihadistischer Internet-Auftritte und -Kommunikatverbreitung.
Where Russian Online Nationalists Go When Their Communities are Banned: A Case Study of Russian Nationalism
2020 Kashpur, V.V., Myagkov, M., Baryshev, A.A., Goiko, V.L. and Shchekotin, E.V. Article
This article presents the results of a case study of the online community Russian Nationalism, one of the most popular Russian nationalists’ online communities on VKontakte in 2012–2016. The article aims to find out where network leaders and common members of nationalist online groups closed on the Russian Internet go and in which thematic communities they continue their activity. Russian Nationalism members’ personal profiles and friendly links on VKontakte were used as data sources. The metrics of social network analysis were used to study Russian Nationalism’s network structure. The study showed that the ban of a massive extreme right nationalist online group leads to an increase in the number of smaller shelter groups that retain their ideological basis, but abandon the clearly expressed hate speech; members of a banned nationalist group flow to groups whose content is both close (militaristic, cultural, and historical) and far (everyday) to the extreme right. The content of the latter can transform in line with the nationalist ideology. The case study of Russian Nationalism shows a high potential for diffusion of extreme right nationalist ideas in the Russian network space, which manifests in their ability to penetrate other thematic and ideological areas of public discourse.
More Grist to the Mill? Reciprocal Radicalisation and Reactions to Terrorism in the Far-Right Digital Milieu
2020 Lee, B. and Knott, K. Article
Reciprocal radicalisation is the theory that extremist organisations are connected and feed on one another’s rhetoric and actions to justify violent escalation. Recent empirical work has suggested that reciprocal radicalisation is a good deal more subtle than is often assumed, and is nuanced by organisational, social and political context. This study seeks to apply the theory of reciprocal radicalisation to the far-right digital milieu, an online space conceptualised as underpinning the varying physical manifestations of the far-right. Based on a qualitative thematic analysis of user posts in three far-right web forums, the study concludes that responses to ideologically opposed terrorism within the far-right milieu are often at odds with the assumed radicalising effects of terrorist attacks. While responses were not uniform, for many users in the far-right digital milieu, jihadist terrorism was an obvious and expected result of the wider failures of politics and society. Although there were some calls for violent reprisal, they were juxtaposed by non-violent responses which interpreted jihadist terror as a consequence and sign of societal decadence and political weakness around issues of migration and rights.
1 2 3 57