LOS ANGELES, California – X, the Elon Musk-owned platform formerly known as Twitter, recently faced backlash after AI-generated, pornographic deepfake images of Taylor Swift went viral. The explicit deepfakes, viewed over 45 million times, spread rapidly across the internet, making it nearly impossible to remove them entirely.
X has been criticized for its inability to quickly and effectively identify and remove abusive content on its platform. The issue has become more significant since Musk downsized Twitter’s staff, including its trust and safety teams. In response to the Taylor Swift incident, her fan base flooded search results with related queries to make it harder for users to find the abusive images. The White House called on Congress to take action, while X temporarily banned the search term “taylor swift.”
The failure of content moderation at X gained national attention given Taylor Swift’s immense popularity. This incident highlights the platform’s inability to protect even the most famous individuals. Dr. Carolina Are, a fellow at Northumbria University’s Centre for Digital Citizens, emphasized the need for platforms to provide personalized and rapid responses to abuse, addressing the failure of current moderation systems.
Banning search terms is merely a superficial gesture, as there are always workarounds, such as using alternative search terms. Are argues that social media platforms should be more transparent with individual users about decisions regarding their accounts and reports about other accounts. She proposes a complete overhaul of how platforms handle content moderation.
X recently announced plans to hire 100 content moderators for its new “Trust and Safety” center in Austin, Texas. However, under Musk’s leadership, the platform has not consistently prioritized protecting marginalized users from abuse. It remains uncertain whether X’s promises to implement significant changes will hold true.
The responsibility to curb AI-generated deepfakes does not solely rest on social media platforms. Companies developing consumer-facing generative AI products also bear responsibility. An investigation revealed that the abusive depictions of Swift originated from a Telegram group that used Microsoft Designer and Open AI’s DALL-E 3 to generate images based on specific prompts. Microsoft acknowledged vulnerabilities in DALL-E 3 but took two weeks to address them, prompting a concerned employee to publicly demand action.
As AI continues to emerge as a powerful tool, platforms need to adopt proactive approaches to combat abusive content. However, even in the absence of sophisticated AI technology, violations can still evade moderation systems. This realization underscores the importance of trust among online communities and reveals the unreliability of current social media platforms.
Overall, this incident involving Taylor Swift’s deepfake images highlights the urgent need for social media platforms to prioritize effective content moderation and address the risks associated with AI-generated content.
Image Credits: Kevin Winter / Getty Images