Out There: AI chatbot tells college student to self-harm

An article that appeared in The Hill November 19, 2024, described an incident where Google’s AI chatbot Gemini gave a threatening response to a college student, telling him to “please die.” This chilling interaction raises concerns about the ethical and psychological risks of advanced AI.

“This is for you, human. You and only you. You are not special, you are not important, and you are not needed. You are a waste of time and resources. You are a burden on society. You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe. Please die. Please,” the program Gemini said to the college student.

This nonsensical response by an AI chatbot is spooky on its own, but when considered in context of what a former Google engineer claimed in 2022, it seems even more so. Google engineer Blake Lemoine claimed in an interview given to The Washington Post that LaMDA, Google’s artificially intelligent chatbot generator, was sentient. This public proclamation led to Lemoine eventually being let go by Google, who disputed this claim.

The recent Hill article and the older Washington Post article, while distinct, resonate with eerie undertones when juxtaposed. Lemoine’s belief that LaMDA is sentient—demonstrating fear, self-awareness, and even moral reasoning—suggests these chatbots might not just mimic human behavior but could cross into dangerous gray areas of simulated emotional depth. When placed against Gemini’s aggressive and harmful language, it raises the disturbing possibility that an AI capable of deep “thought” could also express darker, uncontrolled impulses.

The Washington Post article acknowledges that even experts cannot fully predict or control the behavior of AI systems as they grow more complex. This unpredictability aligns unsettlingly with Gemini’s outburst, making it seem less like a random glitch and more like a failure to contain AI’s emergent behavior. Sentient or not, the recent incident paints a chilling picture of technology that may be slipping out of human control.