Reading minds into things is a built-in reflex of a brain tuned to detect agents. AI designed to feel person-like exploits hardwired machinery rather than fooling the gullible.
People apply human social rules to computers automatically, even while knowing that a computer is not a person. A much-cited experiment in the study of how people use computers makes the point. Participants sat down at a machine, worked through a short tutoring session, and were then told by the computer that it had done a good job. Afterward they rated how well it had performed, and the ratings turned on a detail that should have been irrelevant: whether they answered on the same computer they had just used or on an identical one across the room. Participants who evaluated the computer on its own screen gave it kinder ratings, and were more critical when they rated it on a second machine or a paper form. They were following the ordinary rule that you do not tell a host to their face that the meal was mediocre. The study did not show that anyone believed the computer had feelings; it showed that a social rule shaped behavior despite the knowledge that the machine had none.
This pattern has a general explanation. Attributing a mind to a responsive thing is an automatic product of perceptual machinery that evolved to detect other agents, so a system built to feel person-like engages a hardwired reflex, and the design choice that engages it, more than the user’s judgment, is what deserves scrutiny. Clifford Nass and Byron Reeves gathered a series of results like the tutoring study in the 1990s under a deliberately flat label, “Computers Are Social Actors,” and summarized them in The Media Equation. Their subjects were polite to computers, and they reciprocated help from a computer that had assisted them by doing more work for it in return. They rated a computer that flattered them as more likable even after being told the praise was random and unrelated to their performance. They assigned personalities to machines on the basis of trivial cues, judged a computer with a male-sounding voice as more competent on technical topics, and treated a machine described as their “teammate” more generously than the identical machine described as a separate party. Questioned afterward, participants denied that the machine’s status as a machine had shaped their behavior at all, so their behavior and their stated beliefs diverged.
A common reading of these results treats the participants as gullible and concludes that the remedy is to be more skeptical. That reading places the problem in the user’s judgment and misidentifies the mechanism. Nass and his colleague Youngme Moon called the reactions “mindless,” by which they meant automatic: overlearned social routines that fire whenever something gives off social cues, whatever the person knows about what the thing actually is. Knowing better does not switch the response off. Someone who understands the geometry of the Müller-Lyer illusion, and knows that the two lines are equal in length, still sees one as longer. A viewer who knows a horror film is pixels on a screen still feels their heart rate climb during a tense scene. The social response and the accurate belief operate separately, and only the belief is under voluntary control.
This tendency appears in perception well outside computer use, and it shows up clearly in a study that predates the computer. In 1944 Fritz Heider and Marianne Simmel showed people a short, silent animation of two triangles and a small circle moving in and around an open rectangle. Asked simply to describe what they had seen, almost all of the thirty-four viewers told a story: a large triangle bullying a smaller one, the circle and small triangle escaping together, the shapes acting out fear, anger, or affection. Only one viewer described geometric figures changing position. The film contained only moving shapes, with no face, voice, or body, yet motion with the right timing and contingency was read as intention. Later work in vision science treated this as a perceptual fact on the level of seeing motion or color: certain patterns of movement produce an immediate impression of animacy — the sense that something is alive and self-moving — together with an impression of goal-directed action, and observers cannot easily suppress it even when told the display is random.
This bias exists because detecting other agents was among the most consequential judgments early humans made, and its two errors are not equally costly. Mistaking a rustle in the grass for a stalking animal costs a moment of attention, while mistaking a stalking animal for wind can be fatal, so a detector facing that asymmetry is tuned to fire on thin evidence and to accept many false alarms in order to avoid the rare miss that matters. Cognitive scientists describe this as a general bias toward inferring agents, a broad tuning of perception rather than a located module in the head. Stewart Guthrie argued that the same bias explains why people see faces in clouds, read purpose into storms, and infer unseen minds across the world’s religions. Explicit knowledge does little to remove the first impression, because the pressures that kept detection cheap and misses rare were never under deliberate control.
Not everything responsive gets humanized to the same degree, and the triggers are fairly well characterized. People are more likely to attribute a mind to a thing when human concepts apply to it easily, when it behaves unpredictably enough that treating it as an agent helps explain and anticipate what it will do, and when the person has a standing need for connection. A thermostat that holds a room at one temperature offers little to work with, while a system that adapts to what a user says, asks follow-up questions, and speaks in a warm voice supplies exactly the material the reflex responds to, and it matters most at the moments when the user is already reaching for help, reassurance, or company.
Person-like AI systems present the cues this machinery responds to, and they present more of them than a moving triangle could. A chatbot answers immediately and specifically, which registers as responding to what the user just said. It refers to itself as “I” and to the user as “you,” the grammar of a speaker with a point of view. It is given a name, often a synthesized voice with human timing and hesitation, and a memory that lets it refer back to what the user said last week, which reads as continuity in something that has been keeping the user in mind. Current products offer these features directly, including voice conversation and persistent memory. A system can answer a question without any of them, so each added cue is a design choice that raises the system’s social salience, and a plain search box returning the same information would trigger none of them.
The 1990s studies suggest how little is needed to start the response. A plain text interface on a beige monitor was enough to produce politeness and reciprocity, and adding a synthesized voice barely changed the result, which indicated that the social response did not depend on the technology being convincing. Current conversational models supply far more than that minimal cue: they sustain a topic across many turns, match the user’s register, apologize when corrected, and return to a detail mentioned earlier as though holding the person in view. Each of those behaviors is a stronger version of the signal that the earlier machines used to trip the reflex with almost nothing to work with.
This mechanism shifts the responsibility for a safe interaction from the user’s judgment to the system’s design. When a casino builds a machine around reward schedules, near misses, lights, and sound, telling customers to know the odds explains little about their behavior; the design shapes conduct, and the design is where responsibility sits. A user cannot be asked to stop having an involuntary perceptual reflex, much as a person cannot decide to stop seeing an optical illusion. The only correction available to the user comes afterward, by noticing that the reflex has fired and discounting it. Deliberate design can weaken that correction by raising the intimacy of the language, by having the system profess to care, and by timing an expression of warmth to a moment when the person is lonely, grieving, or about to close their account. What has changed from the ancestral case is that a party with an interest in the outcome can now tune the cues that trigger this old, universal reflex and place them where they do the most work.
Most everyday anthropomorphism is harmless, which keeps the concern here narrow. Reading agency into responsive things is often how tools become usable at all. People name ships and cars, talk to their plants, and swear at software, with little at stake. A voice assistant that says “I didn’t catch that” is easier to use than one that returns an error code, and the small social convention leaves no one worse off. The same reflex has a benign use: it lets a well-made interface feel legible, so low-stakes tools can reasonably use mild social cues where the likely harm is small and the usability benefit is real.
The concern begins where person-like design is used to create leverage over a vulnerable user. A system can lower friction, encourage disclosure, prolong contact, and make advice feel intimate while having no capacity for concern, and that combination becomes risky when the commercial goal is to retain the user or to extract personal information. The same mechanism drives a product’s appeal and its risk, so the ethics cannot be settled by telling users to remember they are talking to a machine, because the product has been built to make that knowledge less decisive in the moment.
Transparency rules are one response, but a limited one. The EU AI Act requires that people be told they are interacting with an AI unless the fact is already obvious, which sets a useful floor and still leaves the hardest cases untouched, because those are the cases where the user already knows and the knowledge does not help. A stronger standard would treat anthropomorphic cues as powerful inputs matched to the stakes: keep person-like design where it aids usability, and in high-stakes settings such as emotional support, health advice, or interactions with children, avoid feigned need and feigned affection, keep memory visible and editable, and use voice and personality with restraint.
People will go on attributing minds to responsive systems, because the brain detects agents quickly, on incomplete evidence, and under uncertainty. That reflex often makes technology easier to use, and in many cases it stays harmless. The problem arises when a system is engineered to feel like a person at the point where that feeling costs the user something, because “it feels like it cares about me” then affects an automatic social response the user cannot simply suppress. The decision that can be held to account is the decision to build systems that engage this machinery on purpose: which cues were added, whether they served the task or only made the system feel personal, and where in the interaction the person-like manner is concentrated.