In this blog, I would like to discuss the role of recommender algorithms as tools for the visibility management of unpleasant content over a social media platform. As said, those algorithms might help reduce the user's exposure to harmful material by way of prioritizing or de-prioritizing the content, but they still give power to inappropriate or hateful content. This is due to the fact that most of these systems are ad-hoc in showing content that might be engaging for the user, but not necessarily wholesome or positive.
Adding to this complexity, such algorithms are also commercially incentivized, which means that the primary motive many times is maximization of user interaction for revenue growth, rather than protection for quality or appropriateness of content. Therefore, the content of any quality that gets a high level of interaction tends to get promoted.
This can make the spread of violent, misinformed, or sexual content really pervasive within the diverse and complex social structure of India. This kind of content usually receives high engagement just for the provoking nature of the content, thus once again through the algorithm, making it popular and relevant. This cycle not only poisons the information ecology but opens quite large questions about the ethical duties of platform operators to provide for safe cyberspace.
A huge challenge in India is the task of content moderation due to the very wide cultural and linguistic diversity of the country. Without this local context—since the content moderation system fails to take it into account—it will surely fail, for it will not get to know the problematic content for specific settings and cultures. This one-size-fits-all approach often leaves much inappropriate content unaddressed because the nuanced context is lost, making these posts seem harmless when they are not.
This further undermines the trust in the reporting mechanisms due to user distrust, hence undercutting effectiveness. Most of the users feel that their concerns are not addressed timely or effectively, if at all. The platforms should make the reporting process transparent, responsive, and contextually aware so that users can trust and rely on these mechanisms.
The urgency, therefore, is in pushing these content moderation technologies to understand and adapt to India's diverse linguistic and cultural landscape. This may not just substantially boost the relevance and safety parameters of the content but also help improve user-generated trust over digital platforms.
Arnika is the Co-founder of Social & Media Matters. She looks after content, policy and user safety issues. Connect with her on LinkedIn.