Disrupting DAESH: Measuring Takedown of Online Terrorist Material and its Impacts
Our findings challenge the notion that Twitter remains a conducive space for Islamic State (IS) accounts and communities to flourish, although IS continues to distribute propaganda through this channel. However, not all jihadists on Twitter are subject to the same high levels of disruption as IS, and we show that there is differential disruption taking place.
• IS’s and other jihadists’ online activity was never solely restricted to Twitter. Twitter is just one node in a wider jihadist social media ecology. We describe and discuss this, and supply some preliminary analysis of disruption trends in this area.
• Our analysis rests on a dataset containing 722 pro-IS accounts (labelled Pro-IS throughout) and a convenience sample of 451 other jihadist accounts (labelled Other Jihadist throughout), including those supportive of Hay’at Tahrir al-Sham (HTS), Ahrar al-Sham, the Taliban and al-Shabaab, active on Twitter at any point between 1 February and 7 April 2017.
• The Pro-IS accounts were located and identified using three methods: the original seed set of accounts (27%) were manually identified by the research team; the second set of accounts (30%) were identified ‘semi-automatically’ (i.e. automatically identified by the system and manually inspected and verified); and the third group of accounts (43%) were identified using an ‘advanced semi-automatic’ system via IS propaganda links.
• For the Pro-IS accounts, 57,574 tweets were collected, with 7,216 (12.5%) of these tweets containing out-links, i.e. links to wider websites, social media platforms, content hosting sites, etc. (not including links within Twitter). For the Other Jihadist
accounts, 62,156 tweets were collected, of which 7,928 (13%) contained out-links.
• One of the overarching objectives of this research was to provide an up-to-date account of the effects of Twitter’s disruption strategy on IS supporter accounts. We found that pro-IS accounts faced substantial and aggressive disruption, particularly those linking to official IS content hosted on a range of other platforms. The majority – around 65% – of the Pro-IS accounts in our dataset were suspended within 70 days of their establishment, with the overall suspension rate of pro-IS accounts probably being considerably higher.
• In a case study of accounts posting links to official IS content in a 24-hour period on 3 and 4 April 2017, 153 accounts were identified. A subset of 50 were ‘throwaway accounts’ (i.e. accounts specifically created on 3 April to disseminate IS propaganda with no expectation that they would stay online for any significant
period of time). Together these accounts sent a total of 842 tweets with out-links to IS propaganda on other online platforms. Within this 24-hour period, 65% of accounts were suspended within the first 17 hours (07.00–00.00 GMT). The 50 throwaway accounts suffered even higher levels of disruption, with a 75% suspension rate during the same time period. This demonstrates that the disruption to official IS propaganda distribution was reasonably effective in the first 24 hours after linking the content.
• We also compared the suspension rates of Pro-IS accounts versus Other Jihadist accounts to check for differential disruption. We found that more than 25% of Pro-IS accounts were suspended within five days of their creation; a negligible number of Other Jihadist accounts were subject to the same rapid response. Of those accounts in our dataset that were eventually suspended (i.e. 455 Pro-IS accounts and 163 Other Jihadist), more than 30% of Pro-IS accounts were suspended within two days of their creation; less than 1% of Other Jihadist accounts met the same fate.
• As a result of this disruption, IS’s ability to facilitate and maintain strong and influential communities on Twitter was found to be significantly diminished. Relationship networks were much sparser for Pro-IS accounts than Other Jihadist accounts. Other Jihadist accounts had the opportunity to send six times as many
tweets, follow or ‘friend’ four times as many accounts and, critically, gain 13 times as many followers as Pro-IS accounts.
• Pro-IS users who persistently returned to Twitter resorted to adopting counter-measures, such as locking accounts, diluting the content of tweets, using innocuous profile pictures, and adopting meaningless Twitter handles. This situation makes it extremely difficult to maintain a strong and influential virtual community.
• Twitter is, however, just one node in a wider jihadist social media ecology. Therefore, we analysed a sample of destinations from Twitter for official IS propaganda at three time points (4–8 February, 4–8 March (excluding 7 March), and 4–8 April 2017). During these periods, Pro-IS accounts linked to 39 different
third-party platforms or content hosting sites, as well as running its own server to host material. Of these, six remained prominent across the three time periods: justpaste.it, IS’s own server, archive.org, sendvid.com, YouTube and Google Drive.
These domains accounted for 83%, 70% and 67% of the URLs in the February, March and April sampling periods respectively. The takedown rate (as of 12 April) was 72%, 66% and 72% for the same sampling periods.
• Only 20 (or 0.04%) of all tweets from Pro-IS accounts contained a telegram.me link. The paucity of such links caused us to explore further; we found that just two of 722 Pro-IS users’ biographies and two of 451 Other Jihadist users’ biographies contained Telegram links. Neither group of accounts was therefore using Twitter to advertise its presence on Telegram.
• Our report makes three recommendations:
1. Modern social media monitoring systems have the ability to dramatically increase the speed and effectiveness of data gathering, analysis and (potentially) intervention, but probably only when deployed in combination with trained human analysts.
2. Active IS supporters who remain on Twitter, in particular content disseminators and their throwaway accounts, could probably be degraded further – though this may have both pros (e.g. detrimental impact on last remaining significant IS supporter Twitter activity) and cons (e.g. further degradation of Twitter as a source of data or open source intelligence on IS).
3. Our focus was largely on Twitter, but we also pointed to the importance of the wider jihadist social media ecology. As our analysis was not restricted to IS users and content, we also underline the often uninterrupted online presence and activity of non-IS jihadists. We point to the usefulness of maintaining a wide-angle view of the online activity of a diversity of these, particularly HTS, across a variety of social media and other online platforms.
• For the future, we propose replicating the present research, but with a larger and more equal sample of HTS, Ahrar al-Sham, and Taliban accounts. This would allow for a more systematic and comparative analysis of the levels of disruption of a range of non-IS jihadists, the vibrancy of their contemporary Twitter communities and Twitter out-linking practices. It would also allow us to identify their other preferred online platforms. Additional research is clearly also warranted into the wider jihadist social media ecology. In particular, we suggest analysing pro-IS and other jihadist activity on Telegram, which is almost certainly where IS’s online community has reconstituted, and comparing this with our present findings.
Research Perspectives on Online Radicalisation
• In recent years, the overwhelming focus of this avenue of research has been on the global jihad movement. This is therefore reflected in the review, but an effort has also been made to highlight similar research on other movements;
• As with the wider debate on radicalisation, there is little agreement on what constitutes online radicalisation and how, if at all, it happens. The influence of online interactions and propaganda on processes of radicalisation therefore remains a highly contested subject. It is a topic that has produced a broad swathe of literature, using different methodologies from a variety of disciplines;
• Consensus is that the Internet alone is not a cause of radicalisation, but a facilitator and catalyser of an individual’s trajectory towards violent political acts; • Use of empirical evidence to draw convincing conclusions remains scarce, and this has negatively impacted on the strength of research on this topic. Nonetheless, the exponential rise in violent extremist use of social media platforms has been the catalyst for an increase in research on the topic, and has begun to provide researchers with new forms of primary source data;
• Extremist use of the Internet has rapidly evolved and effectively adapted to a constantly shifting online media environment. Indeed, organisations – both public and private – that seek to respond to this are still playing catch-up, and have yet to mount a convincing response;
• One of the most celebrated aspects of social media – its ability to tailor content that appears on users’ feeds that appeals to their specific values and interests and plugs them into networks of like-minded individuals – is also what makes it a key asset for extremist groups. Both in the physical and virtual realm, such groups rely heavily upon isolating potential recruits from views and opinions that diverge from their prevailing ideologies and narratives. Extremists seek to insert people into echo chambers that amplify their message and suppress any contrary opinions. Thus, by its very nature, social media creates for its users an environment that, in some cases, is conducive to radicalisation. This is neither a criticism of social media companies nor a call for them to fundamentally change the services they provide, but rather a comment on the complexity of the challenge of online radicalisation;
• While some analysts and scholars call for measures such as censorship, others argue that softer approaches, such as creating online so-called ‘counter-narratives’ and educating Internet users, would be more effective. However, it is clear that there remains both a lack of understanding of how this would occur, or how such narratives could be effectively disseminated. While very few studies provide a convincing explanation of either, there are signs that a more sophisticated approach is beginning to take shape.
Future Trends: Live-streaming Terrorist Attacks
Online Behaviours of Convicted Terrorists
Violent Extremism and Terrorism Online in 2016: The Year in Review
Check the Web - Assessing the Ethics and Politics of Policing the Internet for Extremist Material
the EU Charter of Fundamental Rights – whilst not conducting a legal analysis. It draws where appropriate upon interpretations by the UN Human Rights Committee, UN experts (such as the High Commissioner for Human Rights and special mandate holders), and regional human rights bodies and courts (such as the Council of Europe and the European Court of Human Rights). The report looks at definitions of ‘extremist material’; the types of monitoring and blocking being undertaken by government agencies and the private sector; and considers the roles of these key stakeholders, along with private individuals and civil society groups. It is based on a two-day workshop in January 2015 with thirty expert stakeholders from law enforcement and intelligence agencies, governments and parliaments, civil society, and universities. Short versions of ten papers were presented to stimulate discussion, following an open call for extended abstracts. These are available on the VOX-Pol website: http://www.voxpol.eu/.
The authors conducted seven follow-up semi-structured interviews with stakeholders from law enforcement, industry, government and civil society; and background policy analysis. The first author also co-organised a workshop on privacy and online policing with the UK’s National Crime Agency in March 2015, and participated in three further workshops where the topics of this report were addressed: two on law enforcement use of communications data, and a third at the United Nations on the relationship between encryption and freedom of expression. Both authors are grateful for the assistance of interviewees, co-organisers, and workshop participants.
The report is produced by the EU-funded VOX-Pol Network of Excellence, and takes particular account of the network’s development of semi-automated search for violent online extremist content and deployment of available tools for search and analytics, including text, video, sentiment, etc., currently employed in other domains for analysis of violent online extremist content. The network’s focus 6 CHECK THE WEB is on making these tools freely available for research purposes to academics, but may also extend to others professionally tasked in this area (such as activists and law enforcement agencies). It is also centrally concerned with the ethical aspects of deployment of such tools and technologies.