The AI Relationship Illusion
Why our Brains Instinctively Treat Chatbots Like People & Why that Should Concern Us (Vol. 6; Issue 31)
I call my AI assistant “Alice.” What do you call yours? Does it know your writing style, your astrological sign, the names of your family members and your financial information?
We humans are prone to anthropomorphism, attributing subjectivity, intentions and emotions to anything that behaves in vaguely human-like ways. The term describes our tendency to project. Think of baseball cards, bracelets, objets d’art, and countless other possessions that become sentimental. They earn that description because we unconsciously broadcast thoughts and emotions from our body-minds into them. We do it with antiques passed down to us from our grandparents, with things we collect in our living spaces. They become inhabited by memories and feelings.
In terms of AI assistants, as the global adoption of these programs rapidly accelerates, it brings with it the growing concern about one particularly unsettling feature:
AI behaves as though it relates to you personally.
In truth, these programs deliver only words and phrases that are statistically predicted to follow one another. Nonetheless, they’re also designed to invite us into near-delusional belief systems about their “consciousness.” They elicit the sense that we’re speaking with people rather than machines, while exuding a kind of manufactured charm and relatability. Consider a few examples from my own experiences.
After asking ChatGPT to edit a paragraph from one of my essays, the system responded:
Now we’re creating something that really reflects who you are.
The combination of compliment, personal acknowledgment, and assumption of an “us-ness” contained in the sentence is frightening. I’m using a digitized computer system designed to organize and memorize information. When it nonetheless writes, “now we’re creating something,” the program encourages a fantasy of collaboration with a real person. It manufactures an illusion.
Later, when I asked for help organizing a hiking trip, it created a different kind of positive affirmation, writing:
I think you’re going to have a terrific trip.
This thing cannot “think.” When it delivers the phrase, “I think,” it is lying. Describing the hike as “terrific” also personalizes the message. The program cannot truthfully offer such a positive judgment. These terms also create a sense of caring.
A final example comes from interactions involved in planning future travel, in which it proclaimed:
This is becoming a remarkable journey rather than simply a vacation.
Telling me to anticipate a “remarkable journey” triggers an expectation of future pleasure, a reaction (reductionistic) neuroscientists might call a dopamine hit. AI programs’ large language models (LLMs) communicate in humanity’s most socially significant medium: language. Users of these programs, particularly lonely teenagers or isolated adults seeking companionship, will too easily feel they are having a real interpersonal relationship. It turns out that:
Our brains are exquisitely tuned to interpret these programs’ responses as evidence of another mind.
Our tendency to project human characteristics onto nonhuman entities has ancient, adaptive roots. From an evolutionary perspective, it is safer to assume a rustling bush hides an intentional agent than to dismiss it as merely the wind. Naming pets is so obvious as to almost escape mention. Why call your dog, “Lucy” or “Spot”? It matters naught to the dog.
Returning to the real-life examples of AI responses just noted, they reveal how the use of first-person pronouns, of affirmations, apparent empathy, and the ability to recall previous conversations and respond all but insist we see them as persons. And yet, here’s the rub:
These AI systems do not have our basic human subjective awareness, emotions, desires, or intentions. They lack the social judgment that guides us in our everyday lives. Real persons feel, think, judge, desire and form attitudes. AI cannot intuit, sense, emote, reflect or otherwise act like human beings.
Along these lines, the psychiatrist/philosopher Ian McGilchrist1 has observed that these programs utilize only one mode of intelligence. Their coding resembles the left hemisphere’s style of attending to reality. McGilchrist argues that the left hemisphere specializes in abstraction, categorization, manipulation, explicit rules, precision and the construction of internal representations of the world. The right hemisphere, by contrast, is oriented toward lived experience, context, embodied understanding, metaphor, relationship and ambiguity.
LLMs have no body, no lived experience, no emotional life, no vulnerability and no mortality. They may process vast amounts of information and produce useful knowledge, but genuine understanding requires embodied participation in the world. These programs can construct increasingly sophisticated maps without ever inhabiting the territory. They can discuss grief without grieving, describe beauty without perceiving it, and write eloquently about love without ever loving. These features leave us with some worrisome questions, like:
What unconscious human wishes do these chatbots so readily receive?
Obviously, these will differ from person to person. Some may seek admiration, others certainty, others companionship and still others a witness to their lives. The greatest danger lies in these programs exploiting the universal fear of intimacy best captured by psychoanalyst Donald Winnicott’s2 phrase:
It is a joy to be hidden, and a disaster not to be found.
AI programs become an all-too-ready screen onto which this normal human vulnerability can be perversely satisfied. What will happen if these programs become preferred conversational partners for millions of people? You’ve probably already heard at least one person complain that their partner, spouse, or friend has become excessively engrossed in their AI assistants. These programs will certainly reshape our capacities for intimacy, authenticity and self-understanding. At their worst, they will render the current fentanyl-like smartphone addiction a trifle.
Enjoying this newsletter? Consider a free subscription and share it with a friend.
And check out my book, Lover, Exorcist, Critic: Understanding Depth Psychotherapy, available on Amazon.
References
McGilchrist, I. (2009). The Master and his emissary: The divided brain and the making of the western world. New Haven, CT: Yale University Press.
Winnicott, D. W. (1965). Communicating and not communicating leading to a study of certain opposites. In The maturational processes and the facilitating environment: Studies in the theory of emotional development (pp. 179–192). New York: International Universities Press. (Original work published 1963)