After losing her marketing job due to the pandemic and then gaining 40 pounds, Remi Bader, 25, began spending more time on TikTok. She built up a following by posting about clothing items not fitting her correctly and her struggle to find larger sizes in New York City stores.
But in early December, Bader, who now has more than 800,000 followers, tried on a too-small pair of brown leather pants from Zara, and viewers caught a glimpse of her partially naked butt. TikTok quickly deleted the video, citing its policy against “adult nudity.” It was upsetting to Bader given that her video, which was meant to promote body positivity, was taken down while videos from other TikTok users that appear sexually suggestive remain on the app. “That to me makes no sense,” she said.
Julia Kondratink, a 29-year-old biracial blogger who describes herself as “mid-sized,” had a similarly unexpected takedown on the platform in December. TikTok deleted a video featuring her wearing blue lingerie due to “adult nudity.” “I was in shock, there wasn’t anything graphic or inappropriate about it.”
And Maddie Touma says she has watched it happen to her videos multiple times. The 23-year-old TikTok influencer with nearly 200,000 followers has had videos of her wearing lingerie, as well as regular clothing, taken down. It made her rethink the content she posts, which can be a difficult tradeoff since her mission is body positivity. “I actually started to change my style of content, because I was scared my account was going to either be removed or just have some sort of repercussions for getting flagged so many times as against community guidelines,” Touma said.
Scrolling through videos on TikTok, the short-form video app especially popular among teens and 20-somethings, there’s no shortage of scantily clad women and sexually suggestive content. So when curvier influencers like Bader and Touma post similar videos that are then removed, they can’t help but question what happened: Was it a moderator’s error, an algorithm’s error or something else? Adding to their confusion is the fact that even after appealing to the company, the videos don’t always get reinstated.
Content-moderation issues aren’t limited to TikTok, of course, but it’s a relative newcomer compared to Facebook, Twitter, and others that have faced blowback for similar missteps for years. Periodically, groups and individuals raise concerns that the platforms are inappropriately and perhaps deliberately censoring or limiting the reach of their posts when the truth is far less clear. In the case of the plus-sized influencers, it’s not evident whether they’re being impacted more than anyone else by content takedowns, but their cases nonetheless offer a window to understand the messy and sometimes inconsistent content moderation process.
“It’s frustrating for me. I have seen thousands of TikTok videos of smaller people in a bathing suit or in the same type of outfit that I would be wearing, and they’re not flagged for nudity,” Touma said. “Yet me as a plus sized person, I am flagged.”
For years, tech platforms have relied on algorithms to determine much of what you see online, whether it’s the songs Spotify plays for you, the tweets Twitter surfaces on your timeline, or the tools that spot and remove hate speech on Facebook. Yet, while many of the big social media companies use AI to complement the experience their users have, it’s even more central to how you use TikTok.
But TikTok’s choice to double down on algorithms comes at a time of widespread concerns about filter bubbles and algorithmic bias. And like many other social networks, TikTok also uses AI to help humans sift through large numbers of posts and remove objectionable content. As a result, people like Bader, Kondratink and Touma who have had their content removed can be left trying to parse the black box that is AI.
It remains unclear exactly how TikTok’s systems misfired for Bader, Touma and others, but AI experts said there are ways to improve how the company and others moderate content. Rather than focusing on better algorithms, however, they say it’s important to pay attention to the work that must be done by humans.
