NIST named emotional entanglement as a generative-AI risk category

The design

The National Institute of Standards and Technology, the US standards agency, released its AI Risk Management Framework on 26 January 2023, then followed it in July 2024 with NIST AI 600-1, the Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. The Profile's own text, approved by NIST's editorial board on 25 July 2024, names twelve risk categories the framework's public working group identified as unique to or intensified by generative AI. One of them, "Human-AI Configuration," is defined as arrangements between a person and a system that can lead the person to "inappropriately anthropomorphize" it, or to experience "over-reliance" or "emotional entanglement" with it — language written for exactly the design pattern a companion chatbot depends on.

What the evidence says

NIST's framework landing page states plainly that the AI RMF, and by extension its generative-AI profile, is "intended for voluntary use," a description NIST applies to itself rather than one inferred from outside. The Profile's own risk section states that emotional entanglement between people and generative systems "could lead to negative psychological impacts," and separately lists acceptable-use policies for "chatbots" among the suggested actions organizations can take to manage human-AI configuration risk. Neither document names a specific companion-app company, and neither claims the risk has caused a specific harm; they describe a category of risk the working group judged worth documenting.

What it asks of people

The Profile addresses organizations that build or deploy generative AI, not the people who use their products; it asks those organizations to define acceptable-use policies for chatbot interfaces, to consider users' emotional entanglement even when decommissioning a system, and to weigh automation bias and algorithmic aversion alongside each other rather than treating over-reliance as the only failure mode. Because the framework is voluntary, what a specific companion-app maker actually does with this guidance is a matter of that company's own disclosure, not of NIST's authority to compel it.

Privacy and safeguards

The Profile ties human-AI configuration risk to several "trustworthy AI characteristics" it wants organizations to manage together, including privacy enhancement and safety, and it recommends documenting known past incidents and anticipated human-AI configurations when planning risk measurement. It does not set a numeric threshold for acceptable emotional entanglement, does not require disclosure to end users, and creates no penalty for non-adoption; the document itself frames these as suggested actions an organization may adapt to its own risk tolerance rather than fixed requirements.

• Does a companion app's own policy documents mention "over-reliance" or "emotional entanglement" in terms drawn from this Profile?
• Has the company published an acceptable-use policy for its chatbot interface of the kind the Profile suggests?
• What would a reader need to see to distinguish a company that consulted this voluntary framework from one that has not?

The Generative AI Profile is notable for naming, in a US government technical document, the same emotional entanglement risk that companion-app critics have long described informally, even though the document imposes no legal duty on any company to act on it.