AI and humans collaborate in TikTok's content review process

TikTok's Singapore-based Transparency and Accountability Center (TAC) showcases a sophisticated content moderation system that blends artificial intelligence with human oversight. As social media platforms face increasing pressure to combat misinformation and harmful content, TikTok's approach aims to balance rapid detection with nuanced judgment.

The process begins with AI algorithms scanning uploaded videos, images, audio, and text for potential violations. These systems are designed to detect not just obvious threats like weapons, but also subtler issues such as alcohol consumption, extremist symbols, and self-harm content. Crucially, the AI considers the context of actions; for instance, holding a kitchen knife in a threatening pose triggers a high 'dangerous tool' score, while using the same knife for eating results in a significantly lower score.

When AI encounters content that is difficult to classify, human reviewers step in. Thousands of trust and safety specialists examine these ambiguous cases, considering cultural nuances and intent to determine whether content violates guidelines. This human element is vital for addressing issues like discerning sarcasm from genuine insult or identifying sophisticated disinformation campaigns. In the first quarter of this year, TikTok removed over 184 million videos globally for guideline violations, with AI flagging 96.7% of them before user reports, and 94.4% removed within 24 hours.

Beyond moderation, TikTok actively curates user feeds to prevent 'filter bubbles' and enhance engagement. Unlike platforms that prioritize content from friends, TikTok's recommendation engine focuses on user interests. It starts by presenting eight diverse videos to new users and refines its algorithm based on viewing habits, likes, and shares. To ensure variety, the system intentionally mixes content the user is likely to enjoy (around 60%) with new or unfamiliar content (around 40%), and prevents consecutive videos from the same creator or genre from appearing. This strategy aims to keep users engaged while exposing them to a broader range of perspectives and content.