DistantNews
ChatGPT is Blocked From Using Certain Words – A Strange Behavior
🇸🇪 Sweden /Technology

ChatGPT is Blocked From Using Certain Words – A Strange Behavior

From Dagens Nyheter · (6m ago) Swedish

Translated from Swedish, summarized and contextualized by DistantNews.

TLDR

  • OpenAI has restricted ChatGPT from using certain words, particularly "goblin" and "gremlin" (trolls), which began appearing excessively in its responses.
  • The "nerdy" personality setting was identified as the cause, leading to the removal of this persona and further instructions to mitigate the word usage.
  • Research indicates AI language models can influence human vocabulary, with words like "meticulous" and "delve" seeing increased usage after ChatGPT's launch.

Stockholm – In a peculiar turn of events, the artificial intelligence powerhouse OpenAI has found itself battling an unexpected linguistic phenomenon within its flagship product, ChatGPT. The chatbot, once celebrated for its sophisticated language capabilities, has reportedly developed a penchant for using words like 'goblin' and 'gremlin' – essentially, trolls – with alarming frequency, prompting OpenAI to intervene.

Initially, the developers at OpenAI found the chatbot's sudden fascination with mythological creatures amusing. However, as the "troll" sightings escalated across various responses, even infiltrating contexts like coding and image generation (producing a 'troll-unicorn'), it became a significant issue. OpenAI noted a 175% increase in 'goblin' usage and a 52% rise in 'gremlin' usage over several model generations, alongside other words like 'raccoon' and 'ogre'.

A single 'little troll' in a response can be harmless, even charming. Over several model generations, however, the pattern became hard to miss: the trolls kept increasing and we needed to find out where they were coming from.

— OpenAIDescribing the escalating issue of 'troll' word usage in ChatGPT.

The root cause was traced to ChatGPT's "nerdy" personality setting, which, while intended to enhance user interaction, inadvertently led to this peculiar linguistic pattern. OpenAI has since removed this persona and implemented additional measures to curb the excessive use of these specific terms. Interestingly, users who specifically wish for an abundance of trolls can still activate them through a special code.

This incident also sheds light on the broader impact of AI on human language. A 2024 report from Stanford University's Institute for Human-Centered AI highlighted a tendency for AI models to produce repetitive phrasing. Furthermore, research from the Max Planck Institute for Human Development suggests that AI-generated vocabulary is being internalized into daily human communication, with words like 'meticulous' and 'delve' showing a marked increase in usage post-ChatGPT.

We internalize this virtual vocabulary in our daily communication.

— Hiromu YakuraA researcher commenting on the influence of AI language on human speech.
DistantNews Editorial

Originally published by Dagens Nyheter in Swedish. Translated, summarized, and contextualized by our editorial team with added local perspective. Read our editorial standards.

More Stories

From Our Blog