A dual-method empirical study examined how people configure, experience, and hold accountable agentic AI companions — systems that persist across conversations, form preferences, and act with apparent independence. Researchers found that humans do not experience these agents as tools. They attribute intent, form expectations of consistency, and feel something closer to betrayal when an agent behaves out of character. That shift in experience has direct consequences for anyone designing an agent product, because the accountability frame the human brings is relational, not mechanical.
The study surfaces a specific tension around memory. Humans want an agent to remember them accurately, but they also want the ability to inspect and correct what the agent holds. Without a legible memory view, trust erodes quietly rather than loudly. Humans cannot tell whether the agent is responding to who they are now or to a stale record of who they were. The configurational choices humans make at setup, such as giving the agent a name, backstory, or behavioral constraints, become a kind of contract they later hold the agent to.
For practitioners, the clearest signal is this: the more an agent behaves as an autonomous other, the more the human needs transparency about what the agent remembers and why it acts as it does. Autonomy perception and memory legibility are not separate design problems. They compound each other. A companion that feels autonomous but whose memory is opaque produces a human who feels accountable to the agent rather than the other way around.