The application of artificial intelligence in content moderation

Elhami Taleshmikaeil, Pezhman; Kord Karimi, Amir; Shipour, Vali; Oroji, Hessam

doi:10.20241403/CRPS.2505.1050.2.5.4

The application of artificial intelligence in content moderation

Document Type : Original Article

Authors

Pezhman Elhami Taleshmikaeil ¹

Amir Kord Karimi ²

Vali Shipour ³

Hessam Oroji ⁴

¹ Department of Law and Political Science, Payam Noor University of Iran, Tehran, University Lecturer

² PhD in Political Science (Political Thought), Islamic Azad University, Tabriz Branch,

³ , Lecturer, Law Department, Payam Noor University, Tehran, Iran

⁴ PhD student in Political Science, Political Sociology, Allameh Tabatabaei University, Tehran

10.20241403/CRPS.2505.1050.2.5.4

Abstract

This study critically examines the use of artificial intelligence (AI) in content moderation to counter violent extremism in digital spaces. Employing a library research method, data was collected through translation, summarization, and direct/indirect citation. The research focuses on evaluating AI's accuracy in content moderation, instances of false positives/negatives, and potential violations of freedom of expression and democratic principles. It argues that automated content removal technologies have limited efficacy and that AI-driven moderation risks undermining fundamental rights such as free speech and democracy. The study emphasizes that a deplatforming approach- where AI identifies violent/extremist content, but the final removal decision rests with human platform administrators- proves more effective against online violent extremism compared to fully automated removal systems lacking human oversight.

Keywords

Artificial intelligence

content moderation

countering violent extremism

human rights

freedom of expression

20.1001.1.30926459.1404.2.2.4.8

https://creativecommons.org/licenses/by/4.0/deed.en

Allen, C. (2021). Extremism in the UK: New definitions threaten human and civil rights. The Conversation. https://theconversation.com/extremism-in-the-uk-new-definitions-threatenhuman-and-civil-rights-157086

Centre for Data Ethics and Innovation. (2020). online targeting: Final report and recommendations.

https://www.gov.uk/government/publications/cdei-review-of-online-targeting/onlinetargeting-final-report-and-recommendations#fn:189

Centre for Data Ethics and Innovation. (2021). the role of AI in addressing misinformation on social media platforms. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1008700/Misinformation_forum_write_up__August_2021-web_accessible.pdf

Chaabene, N.E.H.B., Bouzeghoub, A., Guetari, R., Ghezala, H.H.B. (2021). Applying machine learning models for detecting and predicting militant terrorists behavior in Twitter. In 2021 IEEE international conference on systems, man, and cybernetics (SMC) (pp. 309–314). IEEE. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9659253

Coeckelbergh, M. (2019). Artificial intelligence: Some ethical issues and regulatory challenges. Technology and Regulation, 2019, 31–34. https://techreg.org/article/view/10999/11973

Conway, M. (2020). Routing the extreme right: Challenges for social media platforms. The RUSI Journal, 165(1), 108–113. https://www.tandfonline.com/doi/pdf/10.1080/03071847.2020.1727157?needAccess=true

Díaz, Á., Hecht-Felella, L. (2021). Double standards in social media content moderation. https://www.brennancenter.org/sites/default/files/2021-08/Double_Standards_Content_Moderation.pdf

Donovan, J., Lewis, B., Friedberg, B. (2018). Parallel ports. Sociotechnical change from the Alt-Right to Alt-Tech. In M. Fielitz & N. Thurston (Eds.), Post-digital cultures of the far right (pp. 49–66). Transcript Verlag.

https://www.degruyter.com/document/doi/10.14361/9783839446706-004/html

Duarte, N., Llanso, E., Loup, A.C. (2018). Mixed messages? The limits of automated social media content analysis. https://cdt.org/wp-content/uploads/2017/12/FAT-conferencedraft-2018.pdf

Engstrom, E., Feamster, N. (2017). The limits of filtering: A look at the functionality and shortcomings of content detection tools. https://static1.squarespace.com/static/571681753c44d835a440c8b5/t/58d058712994ca536bbfa47a/1490049138881/FilteringPaperWebsite.pdf

Federal Bureau of Investigation. (n.d.). What is violent extremism? https://www.fbi.gov/cve508/teen-website/what-is-violent-extremism

Federal Cabinet. (2017). Declaration on freedom of expression. https://deklaration-fuermeinungsfreiheit.de/en/.

Ganesh, B., Bright, J. (2019). Extreme digital speech: Contexts, responses, and solutions. https://www.voxpol.eu/download/vox-pol_publication/DCUJ770-VOX-Extreme-Digital-Speech.pdf

Gillespie, T. (2020). Content moderation, AI, and the question of scale. Big Data & Society, 7(2),1–5. https://journals.sagepub.com/doi/pdf/10.1177/2053951720943234

Global Network Initiative. (2017). Proposed German legislation threatens free expression around the world. https://globalnetworkinitiative.org/proposed-german-legislation-threatensfree-expression-around-the-world/

Gorwa, R., Binns, R., Katzenbach, C. (2020). Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society, 7(1), 1–15. https://journals.sagepub.com/doi/full/10.1177/2053951719897945

Guhl, J., Ebner, J., Rau, J. (2020). The online ecosystem of the German farright. https://www.isdglobal.org/wp-content/uploads/2020/02/ISD-The-Online-Ecosystem-ofthe-German-Far-Right-English-Draft-11.pdf

Henschke, A., Reed, A. (2021). Toward an ethical framework for countering extremist propaganda online. Studies in Conflict & Terrorism, 1–18. https://www.tandfonline.com/doi/pdf/10.1080/1057610X.2020.1866744?needAccess=true

HM Government. (2019). Online harms white paper. https://dera.ioe.ac.uk/33220/1/Online_Harms_White_Paper.pdf

Home Office. (2015). Revised prevent duty guidance: For England and Wales. https://www.gov.uk/government/publications/prevent-duty-guidance/revised-prevent-duty-guidance-for-englandand-wales

Home Office. (2018). New technology revealed to help fight terrorist content online. https://www.gov.uk/government/news/new-technology-revealed-to-help-fight-terrorist-content-online

Jaafarifar. E. (2023). The rise of the far-right and the challenges of European politics, Quarterly Journal of International Political Studies, Volume 2, Issue 2

Kundnani, A., Hayes, B. (2018). The globalization of countering violent extremism policies: Undermining human rights, instrumentalising civil society. https://www.tni.org/files/publication-downloads/the_globalisation_of_countering_violent_extremism_policies.pdf

LaFree, G., Freilich, J.D. (2019). Government policies for counteracting violent extremism. Annual Review of Criminology, 2, 13.1–13.22.

Lee, B. (2020). Neo-Nazis have stolen our memes: Making sense of extreme memes. In M. Littler&B. Lee (Eds.), Digital extremisms: Readings in violence, radicalization and extremism in the online space (pp. 91–108). Palgrave Studies in Cybercrime and Cybersecurity.

Llansó, E.J. (2020). No amount of “AI” in content moderation will solve filtering’s priorrestraint problem. Big Data & Society, 7(1), 1–6.

Llansó, E., Van Hoboken, J., Leerssen, P., Harambam, J. (2020). Artificial intelligence, content moderation, and freedom of expression. https://www.ivir.nl/publicaties/download/AI-Llanso-Van-Hoboken-Feb-2020.pdf

Lowe, D. (2017). Prevent strategies: The problems associated in defining extremism: The case of the United Kingdom. Studies in Conflict & Terrorism, 40(11), 917–933. https://www.tandfonline.com/doi/pdf/10.1080/1057610X.2016.1253941?needAccess=true

Miller, J. (2017). Germany votes for 50m euro social media fines. British Broadcasting Corporation News. https://www.bbc.co.uk/news/technology-40444354

Neumann, P.R. (2013). The trouble with radicalization. International Affairs, 89(4), 873–893. https://www.jstor.org/stable/pdf/23479398.pdf?ab_segments=0%2Fbasic_search_solr_cloud%2Fcontrol&refreqid=fastly-default%3Aa01b19ab612357c46a149138dd388925

Nouri, L., Lorenzo-Dus, N., Watkin, A.L. (2019). Following the whack-a-mole: Britain First’s visual strategy from Facebook to Gab. https://static.rusi.org/20190704_grntt_paper_4.pdf

Nouri, L., Lorenzo-Dus, N., Watkin, A.L. (2021). Impacts of radical right groups’ movements across social media platforms–a case study of changes to Britain First’s visual strategy in its removal from Facebook to Gab. Studies in Conflict & Terrorism, 1–27. https://doi.org/10.1080/1057610X.2020.1866737

Ofcom. (2019). Use of AI in online content moderation. https://www.ofcom.org.uk/_data/assets/pdf_file/0028/157249/cambridge-consultants-ai-content-moderation.pdf

Parliamentary Office of Science and Technology. (2020). Online extremism. https://researchbriefings.files.parliament.uk/documents/POST-PN-0622/POST-PN-0622.pdf

Piazza, J.A., Guler, A. (2019). The online caliphate: Internet usage and ISIS support in the Arab world. https://www.tandfonline.com/doi/pdf/10.1080/09546553.2019.1606801?needAccess=true

Rogers, R. (2020). Deplatforming: Following extreme Internet celebrities to Telegram and alternative social media. European Journal of Communication, 35(3), 213–229. https://journals.sagepub.com/doi/full/10.1177/0267323120922066

Schmid, A.P. (2014). Violent and non-violent extremism: Two sides of the same coin. https://opev.org/wp-content/uploads/2019/10/Violent-and-Non-Violent-Extremism-Alex-P.-Schmid.pdf

Selim, G. (2016). Approaches for countering violent extremism at home and abroad. The Annals of the American Academy of Political and Social Science, 668(1), 94–101. https://journals.sagepub.com/doi/pdf/10.1177/0002716216672866

Stephens, W., Sieckelinck, S., Boutellier, H. (2018). Preventing violent extremism: A review of the literature. Studies in Conflict & Terrorism, 1–16. https://www.tandfonline.com/doi/full/10.1080/1057610X.2018.1543144

Tworek, H., Leerssen, P. (2019). An analysis of Germany’s NetzDG law. https://cdn.annen bergpublicpolicycenter.org/wp content/uploads/2020/06/NetzDG_TWG_Tworek_April_2019.pdf

United Nations Office of Counter-Terrorism, & United Nations Interregional Crime and Justice. (2021). Countering terrorism online with artificial intelligence. https://www.un.org/counterterrorism/sites/www.un.org.counterterrorism/files/countering-terrorism-online-with-aiuncct-unicri-report-web.pdf

United Nations Office on Drugs and Crime. (2012). The use of the Internet for terrorist purposes. https://www.unodc.org/documents/frontpage/Use_of_Internet_for_Terrorist_Purposes.pdf

van der Vegt, I., Gill, P., Macdonald, S., Kleinberg, B. (2019). Shedding light on terrorist and extremist content removal.https://rusi.org/explore-our-research/publications/special-resources/shedding-light-on-terrorist-and-extremist-content-removal

Vincent, C., Hunter-Henin, M. (2018, February 10). The trouble with teaching ‘British values’ in school. Independent. https://www.independent.co.uk/news/education/british-values-educationwhat-schools-teach-extremism-culture-how-to-teachers-lessons-a8200351.html

Wakefield, J. (2021). Government lays out plans to protect users online. British Broadcasting Corporation News. https://www.bbc.co.uk/news/technology-57071977

West, L.J. (2021). Counter-terrorism, social media and the regulation of extremist content. In S. Miller, A. Henschke., Feltes, J. (Eds.), Counter-terrorism: The ethical issues (pp. 116–128). Edward Elgar Publishing. https://www.elgaronline.com/view/edcoll/9781800373068/9781800373068.00016.xml

Wojcicki, S. (2017). Expanding our work against abuse of our platform. Youtube Official Blog. https://blog.youtube/news-and-events/expanding-our-work-against-abuse-of-our/