TikTok recently introduced an AI-powered text summary feature designed to automatically generate brief descriptions of videos. The intent was to help users quickly grasp the content of a video without watching it, improving accessibility and discoverability. However, the feature quickly became a source of amusement and confusion due to its tendency to produce bizarre, nonsensical descriptions. Within days of its rollout, reports emerged of serious errors, prompting TikTok to heavily scale back the functionality. According to a report from Business Insider, the feature is now limited to identifying products visible in videos, a far cry from its original purpose.

The Rise and Fall of TikTok's AI Summaries

The feature initially appeared on a subset of TikTok users, generating one-line summaries that often missed the mark. The most notable example involved a straightforward video of popular TikTok creator Charli D'Amelio dancing or performing a simple action. The AI described the video as a "collection of various blueberries with different toppings." This absurd error quickly spread across social media, with users sharing screenshots and mocking the AI's apparent hallucinations. Another widely cited error described a video of Shakira as "a repetitive sequence of several distinct blue shapes appearing and moving across the screen." A dog training video was similarly mischaracterized as "a captivating display of intricate origami art, meticulously folded from a single sheet." These examples highlight the AI's inability to correctly interpret visual content, especially when faced with human figures, animals, or complex scenes.

Business Insider also reported other errors, such as describing a video of a person eating food as a "close up of a person using a tool." The pattern was clear: the AI frequently failed to recognize basic objects, actions, and context. The feature's rollout appeared to be in a test phase, but the errors were so severe that TikTok moved quickly to restrict it. The current version of the feature, according to the report, only identifies products seen in the video, such as brand logos or specific items. This limits the potential for hallucination but also greatly reduces the utility of the feature.

Bizarre Errors That Went Viral

The errors were not limited to a few isolated cases. Social media platforms, especially Twitter (now X) and Reddit, saw a wave of posts showcasing the AI's failures. Users compiled lists of the most egregious mistakes. For example:

• A video of a cat playing with a toy was summarized as "a large metal machine in a warehouse."
• A cooking tutorial was described as "a collection of colored lights on a dark background."
• A travel vlog from a beach was summarized as "a series of geometric shapes moving across a grid."
• A makeup tutorial became "a demonstration of a new type of industrial equipment."
• A video of someone doing a handstand was labeled "a group of people standing in a line."

These summaries suggest that the underlying AI model was struggling with visual reasoning, perhaps due to inadequate training data or a flawed approach to simplifying complex scenes into short text. The errors bear a striking resemblance to the well-documented phenomenon of AI hallucination, where models generate plausible-sounding but entirely incorrect information. In this case, the AI was not just wrong about details—it was consistently wrong in ways that revealed a fundamental misunderstanding of the visual world.

Comparison to Other AI Fails

TikTok's AI troubles echo similar issues seen in other major platforms. Earlier in 2024, Google's AI Overviews feature was caught giving dangerously incorrect answers, such as suggesting that eating rocks is healthy or that running with scissors is a great way to get exercise. Google also faced backlash when its AI generated false medical advice. The common thread is that large language models and computer vision models, while powerful, often lack common sense and contextual understanding. TikTok likely leveraged a combination of vision models and language models to generate these summaries. The errors indicate that the vision model misidentified objects, and the language model then confidently constructed a plausible-sounding but false description based on those misidentifications.

Similarly, other social media platforms have experimented with AI-generated content summaries. YouTube has used AI to generate video descriptions and captions, but those features are typically more accurate because they rely on audio transcripts rather than visual interpretation. TikTok's approach was more ambitious—trying to describe the visual content itself—and it failed spectacularly.

The problem is not unique to TikTok. Many AI-driven features have faced backlash after public rollouts. Meta's AI stickers, introduced in 2023, occasionally generated inappropriate or offensive images. Microsoft's Bing Chat (now Copilot) initially suffered from erratic behavior and was quickly restricted. The pattern suggests that tech companies are pushing AI features to the public faster than the technology can reliably deliver, often at the expense of user experience.

What This Means for AI in Social Media

TikTok's decision to scale back the feature is a pragmatic response, but it also highlights the challenges of integrating AI into consumer-facing products. The company has not provided an official statement on the matter, though Mashable reached out for comment. The lack of communication may be intentional; companies often prefer to quietly roll back problematic features rather than draw attention to failures. However, the viral spread of the error examples makes it difficult to ignore.

The incident raises important questions about the trustworthiness of AI systems. For TikTok, which relies on user-generated content and personalized recommendations, inaccurate summaries could mislead users and degrade the platform's credibility. The feature was presumably intended to improve accessibility for users who are visually impaired or prefer text-based navigation. Instead, it produced confusion and amusement. It also underscores the need for rigorous testing before public deployment. TikTok likely tested the feature internally, but the errors suggest that the testing did not cover a sufficiently diverse set of videos or scenarios.

Looking ahead, it is possible that TikTok will re-launch a more accurate version of the feature after further refinement. The company has invested heavily in AI, including its recommendation algorithms and creative tools. The text summary feature may be revived with better models, perhaps using multimodal AI systems that are trained on larger and more varied datasets. Alternatively, TikTok could opt to rely on user-generated captions or transcripts instead of generating summaries from scratch. The platform already allows creators to add text descriptions, and those could be used as a fallback.

Ultimately, the rollout and subsequent retreat of TikTok's AI summaries serve as a cautionary tale. AI can perform remarkable feats, but it is also prone to unpredictable failures. When these failures affect millions of users, they can undermine trust in both the technology and the company. The path forward requires a balance between innovation and caution. As AI becomes more integrated into everyday apps, we will likely see more examples of both triumphs and blunders. TikTok's blueberries-and-origami episode is just one more chapter in the ongoing story of artificial intelligence in the wild.

Source: Mashable News