Stalking, doxxing, slut-shaming. You don’t necessarily need to know the taxonomical distinctions between these terms if you are a woman (or a girl, unfortunately) who is fairly active online, because it is likely that you would have experienced some form of this. The Economist Intelligence Unit found in a study that 85% of women have either personally experienced gendered digital violence or have been witnesses to such crimes. The online hate directed at the wives of cricketers, the relentless body shaming and slut-shaming of actresses, and the constant abuse aimed at feminist public figures have been etched into our collective digital memory. But why should we be concerned? Every celebrity receives hate online, right? With the advent of AI and LLMs, we are seeing an unprecedented expansion of digital access: more people than ever before are plugged into infrastructures that connect them instantaneously to the world. It is tempting to assume that with such mass connectivity, a few “bad actors” will inevitably run amok and that online hate is simply a by-product of scale. Well, no.
The problem is that the hate towards women is not accidental or neutral. It is systemic – the digital expression of long-standing socio-cultural hierarchies. The internet does not create misogyny; it encodes it, because it is built upon and embedded within social structures that have historically normalised the policing, shaming, and disciplining of women. When we fail to detect and filter gendered hate in digital spaces, two problems emerge. First, it reveals that our existing systems were never designed to recognise this hate as a problem in the first place, because they were programmed within social contexts where such hostility is normalised rather than challenged. Second, this failure ensures that future systems will inherit and reproduce these same biases, encoding gendered hate into the very foundations of technological infrastructures.
Epistemic bias in AI models
AI is increasingly used for content moderation in social media platforms. Research on automated hate speech detection is growing, with ongoing efforts to fine-tune models for effectiveness. However, models still struggle to distinguish toxic content from neutral content. This becomes more complicated when the language used for hate isn’t English or other Western languages (Narayanan and Kapoor, 2024). Most Indian languages are considered “low-resource languages” that do not have an adequate amount of organised data capable of training machine learning or deep learning models (Nandi et al.). As a result, harmful content in these languages often remains undetected, while harmless expressions are sometimes misclassified. This creates uneven layers of visibility and protection across linguistic groups.
From an epistemic standpoint, these gaps reveal how AI systems are built upon existing global hierarchies of knowledge production. Languages with robust digital corpora receive better safeguards, while speakers of low-resource languages remain more vulnerable to unchecked hate, misclassification, and platform-level neglect.
When we fail to detect and filter gendered hate in digital spaces, two problems emerge. First, it reveals that our existing systems were never designed to recognise this hate as a problem in the first place, because they were programmed within social contexts where such hostility is normalised rather than challenged. Second, this failure ensures that future systems will inherit and reproduce these same biases, encoding gendered hate into the very foundations of technological infrastructures.
Another layer of complexity comes from the way people actually speak and write online. Kapoor et al. (2018) note that research on code-mixed Indian languages is surprisingly scarce, despite the fact that Indian social media users constantly switch between English and an Indian language in everyday conversation. One of the most widespread forms of this switch is Hinglish, the Hindi-English blend that dominates online communication. They document that this is partially due to the following reasons: (i) Hinglish consists of no fixed grammar and vocabulary. It derives a part of its semantics from Devnagari and another part from the Roman script. (ii) Hinglish speech and written text consist of a concoction of words spoken in Hindi as well as English but written in the Roman script. And this is all happening in Hindi – a majority language that already receives far more NLP investment than most other Indian languages. Helm et al. (2024) highlight language modelling bias in AI development, which is currently limited to three percent of the world’s most widely spoken, financially and politically backed languages. For a country like India, with its vast landscape of languages and dialects, this presents a major challenge: most Indian languages remain underrepresented or entirely absent in the datasets and benchmarks that shape today’s AI systems.
Why computational problems demand intersectional thinking
Targeted hate online doesn’t happen in a vacuum; it operates within broader social, political, and technological contexts that shape its emergence and impact. Hate speech is taken to express hostility to and about historically and contemporarily oppressed groups, and, in so doing, it vilifies, degrades, discriminates, maligns, and so on (Richardson-Self, 2018). It is not simply that the speech in question “picks out” an oppressed group; it is that such speech enacts, and thus reinforces and perpetuates oppression (McGowan 2009, 404–405).
Hate in the Indian digital sphere, especially gendered hate, can be vastly different from Western contexts. In India, gendered hate sits on a unique bed bolstered by norms on female seclusion, patrilocality, religion, and caste. This is further worsened by the intersection of caste and religion-based endogamy and associated honour. In her groundbreaking work, The Next Billion Users (Arora, 2019) talks about how navigating the digital space is uniquely challenging for women from the Global South, balancing the newfound opportunity to engage in digital agency while having to always be on the lookout lest their and their family’s “honor” is tarnished. The digital world becomes a space of opportunity and surveillance at the same time, where every act of expression carries the risk of social scrutiny, moral policing, and reputational harm.
A 2014 study of young Indian Muslim women revealed the challenges of disclosing visual aspects of themselves (in digital spaces), given that their physical modesty is closely associated with the upholding of their family honor. In the West, most young single women choose their most attractive photos for their Facebook profiles to enhance their romantic prospects (Mishra & Basu, 2014) (Arora, 2019). “It is not that simple for young Indian Muslim women. Much deliberation goes into selecting a modest photo. A bad choice in the wrong hands could have devastating consequences for the woman and her family. In 2013, two Indian Muslim clerics issued a fatwa against young Muslim women uploading their photographs on social networking sites such as Facebook. Shortly after that, however, another cleric declared that this fatwa was un-Islamic, positing that as long as it brought about good communal bonding, sharing was indeed caring.”
Hate in the Indian digital sphere, especially gendered hate, can be vastly different from Western contexts. In India, gendered hate sits on a unique bed bolstered by norms on female seclusion, patrilocality, religion, and caste. This is further worsened by the intersection of caste and religion-based endogamy and associated honour.
As observed in the disturbing case of two high-profile incidents of gendered and communal online hate in India, Muslim women were targeted through mock “auctions” on GitHub platforms (The Times of India, 2022). In July 2021, the Sulli Deals app circulated images of Muslim women labeled as “deal of the day” without consent. Despite widespread outrage, arrests were delayed until early 2022. A similar case followed on 1 January 2022 with the Bulli Bai site, which hosted doctored images of Muslim women, including journalists and activists, again for “auction.” This is just one of the horrifying examples of how religious discrimination and gender intersect in the Indian context to create a unique space for online gendered hate.
Terms such as “nachniyan”, “mujra”, and “kaluvi” or “kali” are frequently deployed in Indian digital spaces to degrade, hypersexualize, or caste-mark women, particularly Dalit and Muslim women. These terms carry deep socio-historical baggage and reflect region-specific modalities of hate that cannot be adequately captured through Western-centric vocabularies or generalized categories such as “sexist” or “abusive.” Similarly, Bhardwaj et al. (2020) note an example of implicit hostility in Hindi of calling someone ‘meetha’, which literally means ‘sweet’ in Hindi; however, the intended meaning in a hostile post could be a derogatory term towards the LGBT community used extensively in Hindi social media discourse.
Way forward
The inability of standard datasets and detection systems to annotate or process such indigenous intersectional forms of hate speech underscores the urgent need for linguistically grounded, culturally situated taxonomies. Without the incorporation of explicit indigenous vocabulary and lived experience, hate speech detection in the Indian context remains partial, exclusionary, and epistemically unjust. Epistemic bias in AI is not simply a technical flaw but a socio-political issue about whose knowledge shapes our digital future. Addressing it requires not only better datasets or algorithms but also structural changes in how we define knowledge, whose experiences we centre, and how we distribute epistemic authority in a world increasingly mediated by AI systems.
When historical marginalisation is combined with epistemic bias, it leads to a unique hotbed where women and girls are doxxed, stalked, and shamed. Words are not neutral; they carry social meaning and power, and in computational systems like AI, this is encoded, analyzed, and used in knowledge production. Olimpia Coral Melo Cruz’s reflection on digital violence illustrates how misogynistic social structure is linguistically encoded:
“I searched the Internet for what was going on and read it was called ‘revenge porn.’ I felt even more guilty, because if it is ‘porn,’ I must have provoked it, and if it is ‘revenge,’ I must have done something to deserve it.”
References:
Arora, P. (2019). The next billion users: digital life beyond the West. Cambridge, Massachusetts Harvard University Press.
Bhardwaj, M., Akhtar, M. S., Ekbal, A., Das, A., & Chakraborty, T. (2020). Hostility Detection Dataset in Hindi. ArXiv (Cornell University). https://doi.org/10.48550/arxiv.2011.03588
Helm, P., Bella, G., Koch, G., & Giunchiglia, F. (2024). Diversity and language technology: how language modeling bias causes epistemic injustice. Ethics and Information Technology, 26(1). https://doi.org/10.1007/s10676-023-09742-6
McGowan, M. K. (2009). Oppressive Speech. Australasian Journal of Philosophy, 87(3), 389–407. https://doi.org/10.1080/00048400802370334
Mishra, S., & Basu, S. (2014). Family honor, cultural norms and social networking: Strategic choices in the visual self-presentation of young Indian Muslim women. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 8(2). https://doi.org/10.5817/cp2014-2-3
Nandi, A., Sarkar, K., Mallick, A., & De, A. (2024). A survey of hate speech detection in Indian languages. Social Network Analysis and Mining, 14(1). https://doi.org/10.1007/s13278-024-01223-y
Narayanan, A., & Kapoor, S. (2024). AI Snake Oil. Princeton University Press.
Richardson-Self, L. (2018). WomanHating: On Misogyny, Sexism, and Hate Speech. Hypatia, 33(2), 256–272. JSTOR. https://doi.org/10.2307/45153688
The Times of India. (2022, January 2). Bulli Bai app controversy: All you need to know | India News – Times of India. The Times of India. https://timesofindia.indiatimes.com/india/bullibai-app-controversy-all-you-need-to-know/articleshow/88647836.cms
