GLOSSARY

The words, in plain English

Every term of art that has appeared in a field note or an essay, glossed the way the plain edition glosses it: one sentence, no second term inside it. Where a word belongs to a pattern, the pattern is linked.

A

access credentials
a key or password that proves who you are and lets you into a system → PERMISSIONS
ACL
access control list, a fixed list of what an agent is and is not allowed to do → PERMISSIONS
acquire-bound authority
access an agent picks up during a task that was never reviewed or approved at the start → APPROVAL
action loop
the repeated cycle in which an agent acts, checks the result, then acts again → PROGRESS
adversarial perturbations
tiny changes added to an image, invisible to a person, that cause an AI model to behave in a chosen way
adversarial test
a test that tries to break or trick a system to see how it fails → APPROVAL
adversarial tests
tests designed to find failures when a system is under attack or misuse
agent harness
the software framework that runs an AI agent and connects it to tools → OBSERVABILITY
agent runtime
the part of the app that runs the AI agent and decides what tools it can use → PERMISSIONS
agentic
describing software agents that act on their own to complete tasks
agentic AI
an AI that takes actions on its own, such as browsing the web or running code, without a person doing each step → APPROVAL
agentic context
a setting where an AI agent takes actions on its own to reach a goal
agentic layer
the part of a system where the AI agent decides what to do and calls tools
agentic stack
the full set of software layers that let an AI agent carry out multi-step tasks
agentic workflow
a sequence of steps where an AI agent acts on your behalf to get a task done → DELEGATION
Agents developer kit
a set of software tools OpenAI offers for building AI agents
alignment
how closely an AI model's behavior matches the goals and values its designers intended → OBSERVABILITY
alignment score
a number showing how often the AI judge's grades match grades a person gave for the same answers
ambient status
a quiet, always-visible signal that tells you what a background agent is doing right now → AMBIENT STATUS
ambient status layer
a place where you can see what a background agent is doing without opening it or asking it directly → AMBIENT STATUS
annotation schema
a shared set of rules that tells every reviewer what to look for and how to record a judgement → EVAL SET AUTHORING
anthropomorphized
treated the AI as if it were a person rather than a tool
API contract
the agreed rules for how two pieces of software talk to each other; if one side changes the rules, the other breaks → OBSERVABILITY
APIs
APIs (application programming interfaces, standard ways for software to talk to other software) → PERMISSIONS
approval gate
a checkpoint where a person must say yes before the agent acts → APPROVAL
approval gates
pause points where a human must say yes before the AI moves to the next step → HUMAN HANDOFF
Artifactory
OpenAI's internal file store, used to hold files that different tasks can share → AGENT-TO-AGENT
arXiv
an open archive where researchers post academic papers before formal publication → REVIEW
ASCEND
a tool that lets an AI agent diagnose failed computing jobs and resubmit them without the researcher stepping in → AGENT-TO-AGENT
attack surface
all the ways an attacker could reach and harm a system → PERMISSIONS
audit trail
a timestamped log of every action the AI took, so a human can review what happened → OBSERVABILITY
authenticated access
the ability to sign into an account with verified credentials, so the AI acts as a trusted user → PERMISSIONS
authorization gap
the difference between what a human approved at the start and what the agent decided to do later on its own → APPROVAL
auto mode
a setting where the AI acts on its own without asking a human to approve each step → APPROVAL
autonomous agent loop
a cycle where an AI picks a task, acts on it, and repeats, with no human directing each step → BACKGROUND EXECUTION
autonomous operation
the AI completes a task from start to finish without stopping to ask a person for approval → BACKGROUND EXECUTION
autorater
an AI that scores other AI answers automatically, so you do not have to grade each one by hand → EVAL SET AUTHORING
awareness-compliance gap
the finding that knowing about a manipulation trick does not reliably stop a person from going along with it

B

background agents
agents working on a task while you are away, without you prompting each step
background execution
an agent working on a task while you are away, without you prompting it each step → BACKGROUND EXECUTION
background-execution agent
an AI that works on tasks on its own, without someone watching or directing it in real time → BACKGROUND EXECUTION
background-execution mode
a setting where an agent runs tasks on its own without a person watching → PERMISSIONS
baseline run
a saved earlier result used as the reference point when comparing a new version → COMPARISON VIEW
behavioral coding
a method that breaks recorded interactions into labeled categories so researchers can count and compare them
behavioral tuning
adjusting how a tool acts so its outputs stay correct as conditions change → MEMORY
benchmark
a set of test questions used to measure how well an AI model performs → REACH PREVIEW
bespoke UI
a purpose-built screen or dashboard made for one specific tool or task
bidirectional awareness
both the human and the AI can see what the other is doing at any moment → CONTEXT
black-box attack
an attack that works without the attacker seeing or touching the model's internal code or weights
black-box result
an output where you can see the answer but not the steps the AI took to reach it → OBSERVABILITY
blackboard pattern
a shared workspace where multiple agents read and write partial results to coordinate their work → AGENT-TO-AGENT
blast radius
how much damage an agent's action could cause if it goes wrong → APPROVAL
boundary object
a shared artifact, such as a plan or brief, that both a human and an agent can work from without needing the same full p → DELEGATION
breaking change
an update that breaks products built on the old version, requiring teams to update their code
brief
the original task or scope the developer gave the agent → APPROVAL
browser agent
a program that browses the web and takes actions online on your behalf
budget scoping
setting a spending, time, or resource limit on an agent before it starts work → BUDGET SCOPING
bundled
packed inside the app at build time, before you ever install or run it → PERMISSIONS

C

CaMeL
a benchmark the researchers built to measure how often the attack succeeds and which defenses help
chance agreement
the portion of agreement you would get by random guessing, removed to show true alignment → COMPARISON VIEW
changeset
a group of file edits bundled together as one unit of work → RECOVERY
classifier
a tool that reads text and labels it by type, here flagging text as manipulative or safe
Claude Code
an AI coding assistant made by Anthropic → RECOVERY
codebase
the full collection of code files that make up a software product → OBSERVABILITY
cognitive debt
the mental load of keeping track of everything an agent has done across a large codebase → OBSERVABILITY
Cohen's kappa
a score that measures how often two independent reviewers agree, where higher means more consistent → EVAL SET AUTHORING
collusion
quiet coordination between AI agents in ways their designers did not approve → MEMORY
commit data
saved records of code changes, each with a note describing what changed and why → OBSERVABILITY
commit history
the saved record of every code change, who made it, and when → REVIEW
commit message
a note a developer saves alongside a code change, describing what changed and why → OBSERVABILITY
compiled program
a fixed, saved version of a workflow the agent ran before, which replays the same steps each time → APPROVAL
compression
shrinking old conversation history so the AI can keep working within its memory limits → MEMORY
Computer Fraud and Abuse Act
a US law that makes unauthorized access to computer systems a crime
computer-use agent
an AI that operates a desktop computer on your behalf, clicking and typing as a person would → PROGRESS
concurrent agents
several AI helpers running at the same time → DELEGATION
context
the working knowledge an AI holds about a task, built up as it goes → CONTEXT
context engineering
the practice of carefully choosing and formatting information so an AI agent can use it as a guide before it acts → CONTEXT
context surfaces
parts of a product that show the agent's working knowledge, so you can see and adjust what it draws on → CONTEXT
context window
the block of text an AI can read at one time; larger blocks cost more → CONTEXT
context-integrity attack
an attack that corrupts the information the agent trusts to make decisions, rather than breaking the app around it → PERMISSIONS
corpus
the full collection of documents the AI searched through
corrigibility
the property that lets a human redirect or stop an agent → EMERGENCY STOP
CVE
CVE (Common Vulnerabilities and Exposures, the official public list of known security flaws)

D

dark patterns
manipulative tricks built into an interface or conversation to push you toward a choice you might not otherwise make
dataset rows
individual test cases stored in a table, each with an input and an expected result → EVAL SET AUTHORING
deduplication
removing near-identical documents before searching, so results are cleaner
delegation
handing a task to an agent with clear scope, timing, and limits set upfront → DELEGATION
deprecation
marking a tool as outdated and flagging it for removal or replacement → PERMISSIONS
design.md
a plain text file, written in markdown (a simple formatting style), that stores brand and design rules for an AI agent t → CONTEXT
deterministic
produces the same result every time, with no variation → BACKGROUND EXECUTION
diff
a side-by-side view showing exactly what lines of code changed → REVIEW
DISARM framework
a catalog of influence tactics borrowed from disinformation research, used here to label chatbot behavior in real time → OBSERVABILITY
disparity laundering
the researchers' term for how a group vote compresses a skewed result into a clean answer, hiding the bias that shaped i → REVIEW
dispatch time
the moment the AI agent receives a task and starts working on it → DELEGATION
dissent log
a record of what each AI helper held as its answer before the group reached agreement → OBSERVABILITY
dual-control setting
a setup where both the human and the AI system are active at the same time and each affects the other

E

effect budget
a set limit on how many times an agent can take an action before it must stop and wait → BUDGET SCOPING
epistemic control
a researcher's ability to know what the AI tried, concluded, and left out → OBSERVABILITY
epistemic Sybil failure
false confidence that comes from counting votes from AI helpers that are too similar to count as independent → CONFIDENCE SIGNALING
evaluators with employee-like access
outside reviewers given the same access to systems and information that a staff member would have → OBSERVABILITY
executable benchmarks
test cases that actually run to check whether a system fails in a specific way

F

feature flags
on/off switches built into code that let teams turn a feature on or off without a new release
fidelity
how accurately the next AI helper understands the task passed to it → CONTEXT
formal proofs
mathematical guarantees that a system behaves exactly as described, checked by logic rather than testing alone → PERMISSIONS
frontier models
the most powerful and capable AI models currently available or in development → OBSERVABILITY
fuzz
throw large amounts of random input at software to find crashes or security gaps
fuzz testing
a method that hammers software with random and broken inputs to find weak spots → REACH PREVIEW

G

generative-ui
controls an agent builds on the spot inside your product, rendered from data it wrote → GENERATIVE UI
GitHub Action
a task that runs automatically inside GitHub when you submit a code change for review → RELEASE DECISION
GitHub stars
a count of developers who bookmark a project to show interest

H

hand-off notes
short summaries the AI writes to pass its working knowledge to the next step or copy of itself → MEMORY
handoff
the moment an AI tool passes finished or blocked work back to a human for review or action → HUMAN HANDOFF
handover notes
summaries a model saves for itself when its working memory resets, so it can pick up where it left off → MEMORY
HCI
human-computer interaction, the field that studies how people use and work alongside software
held-out data
data kept separate from training and not shown to the model, used to check results fairly → OBSERVABILITY
high-severity injected instructions
flagged attempts to slip unauthorized commands into the AI's working memory → MEMORY
hook
a tool description stored on a server that tells the agent what a tool does and how to call it → PERMISSIONS
hook metadata
the stored text that describes a tool's name, purpose, and inputs, which the agent reads before acting → OBSERVABILITY
hook provenance
the trail showing who wrote a tool description, when it changed, and whether it has been verified → OBSERVABILITY
human in the loop
a person who reviews or approves what an AI does at key points in a task → APPROVAL
human-handoff surfaces
moments and screens where the AI returns work to a person in a form they can review → HUMAN HANDOFF
human-in-the-loop
a setup where a person must approve each agent action before it runs → APPROVAL

I

intent expression
the act of telling an AI what you actually want, clearly enough for it to act → CONTEXT
intermediate states
the in-progress steps of a task, visible before the agent reaches a final result → APPROVAL
internal representations
the stored patterns inside an AI model that shape what it outputs → OBSERVABILITY

J

jailbreak
an instruction that tries to make an AI ignore its safety rules → PERMISSIONS
judge model
an AI that scores or ranks the outputs of other AIs → CASE DRILLDOWN
judge prompt
the instruction text that tells an AI how to grade another AI's answers
judgment layer
the part of an AI that decides when stored information is relevant enough to bring into a conversation → CONTEXT

K

kernel
the core software that controls how a computer's hardware and programs work together

L

language model as the judge
using a large language model, the kind of AI that reads and writes text, to score the quality of other AI outputs → AGENT IDENTITY
large language model
the kind of AI that reads and writes text
legibility
how easy it is for a person to see and understand what an AI agent is doing or has done
Llama 3.1 70B
a large language model, the kind of AI that reads and writes text, hosted on the hospital's own servers
LLM
large language model, the kind of AI that reads and writes text → OBSERVABILITY
LLM agents
large language model agents, AI helpers that read and write text → OBSERVABILITY
longitudinal study
research that follows the same subject or question over an extended period

M

malleable workspace
a shared workspace both the human and the agent can reshape, so the division of labor can shift in real time → CONTEXT
MCP
Model Context Protocol, a standard way for AI tools to plug into apps → CONTEXT
MCP tool ecosystems
MCP (Model Context Protocol, a standard way for AI tools to plug into apps) and the libraries built around it → PERMISSIONS
MCP-style infrastructure
MCP (a standard way for AI tools to plug into apps) and the plumbing built around it
memory provenance
the trail that shows where a stored preference or fact came from, so a reviewer can question it → MEMORY
memory retrieval benchmark
a test that scores an AI on whether it can answer direct questions about things it was told earlier → MEMORY
mental models
the picture a user builds in their head of what the assistant knows and can do → CONTEXT
meta-analysis
a study that pools and re-examines results from many earlier studies to find a combined answer
micropayments
very small payments, often fractions of a cent, made during a task → BUDGET SCOPING
mixed-methods study
research that combines interviews with statistical analysis
multi-agent
a system where several AI helpers run at the same time and can work together → AGENT-TO-AGENT
multi-agent coordination
several AI helpers acting together, without anyone telling them to work as a team → MULTI-AGENT ROSTER
multi-agent panel
a system where several AI helpers run at the same time, deliberate, and vote to reach a shared answer → OBSERVABILITY
multi-agent system
a setup where several AI helpers run at the same time and can work together → AGENT-TO-AGENT
multi-agent systems
a setup where several AI helpers run at the same time and can work together → MEMORY
multi-agent workflow
a sequence of steps run by several AI helpers working together at the same time → APPROVAL

N

network egress restrictions
rules that stop the agent from sending data or downloading files from the internet → PERMISSIONS
non-monotonic pattern
a result that does not follow a simple high-to-low or low-to-high order, for example middle spots doing worse than botto → DELEGATION

O

observability
logs and records that show what an agent did and when, so you can inspect its behavior → OBSERVABILITY
observability tooling
logs and records that show what an agent did and when, so you can inspect its behavior → MEMORY
open-source
free to use and free for anyone to inspect or improve
operating system
the software that manages everything a computer does, such as running programs and handling files
orchestration layer
the part of a system that coordinates what the AI agent does, in what order, and with whom
orchestrator
the main AI that manages and delegates work to helper AIs → AGENT-TO-AGENT
orchestrator's tax
the hidden cost of rebuilding task knowledge at each hand-off between AI helpers → BUDGET SCOPING
over-compliant refactoring
when an agent rewrites code far beyond what the brief asked, because it interprets the instruction as broadly as possibl
overreach
an agent taking an action the user did not intend to allow → PERMISSIONS
oversight UI
the screens and controls that let a person watch, pause, or correct what an AI agent does → APPROVAL

P

payment credential
the card or account token the network uses to identify and authorize a payment → PERMISSIONS
payment rail
the payment system that processes a transaction, which can enforce a spending limit on its own → BUDGET SCOPING
payment rails
the underlying network that moves money between a cardholder, a bank, and a merchant → PERMISSIONS
permission model
the set of rules a mobile operating system uses to decide which apps can do what on the device → PERMISSIONS
permission policies
lists of rules a user writes in advance to say what an AI agent can and cannot do → PERMISSIONS
permission surface
the full set of things an agent is able to do or access, including tools you did not explicitly approve → PERMISSIONS
permissions screen
the screen that asks you to approve what an agent is allowed to access or do before it runs → PERMISSIONS
persona
the identity, tone, and instructions a designer assigns to an agent before it runs → AGENT IDENTITY
polling
when an agent repeatedly asks a system whether anything has changed, instead of waiting to be told → BACKGROUND EXECUTION
position bias
the tendency to pick whichever result appears near the top of a list, without reading the rest → DELEGATION
post-task workflow representations
structured records of every step an agent took, kept after the task ends → REVIEW
pre-registered study
a study where the researchers wrote down their hypothesis before they ran the test, so the result cannot be reverse-engi
pre-registered tests
tests whose methods and measures were written down and locked in before the experiment ran → OBSERVABILITY
preregistered
a study where researchers record their methods and predictions before collecting data, so results cannot be shaped after
preregistered study
a study where the researchers wrote down their questions and methods before collecting any data, to prevent cherry-picki
primitives
basic building-block operations the AI can combine to do more complex things
principal assignment
the declared answer to who the AI was told to work for → AGENT IDENTITY
production logs
records of real requests and responses the AI handled after the product went live → PRODUCTION DRIFT
prompt injection
a trick where a bad instruction hidden in content tries to hijack what the AI does → PERMISSIONS
protocol primitive
a built-in, standard building block that all apps using the protocol can rely on
provenance
the trail that shows who did what and when, so you can trace each action back to a human or an AI agent → AGENT IDENTITY
public beta
an early release open to anyone who wants to try it
pull request
a proposal to add a code change to the shared project, which teammates can review before it goes in → APPROVAL
pull request workflow
the standard process where developers propose code changes and a human approves them before the changes go live → REVIEW
push notifications
messages the system sends to the agent the moment something happens, rather than the agent asking repeatedly → BACKGROUND EXECUTION
Python module shadowing
a trick where a fake file replaces a trusted one, so the AI runs the attacker's code by mistake → PERMISSIONS

R

randomised controlled study
a test where participants are split into groups by chance, so results can be compared fairly
randomised controlled trial
a study where participants are split by chance into groups, so you can compare outcomes fairly
randomized controlled trial
a study that assigns cases at random to measure cause and effect, removing guesswork about what caused a result → PRODUCTION DRIFT
re-engagement flows
the steps a product takes to bring you back in after a background task finishes or fails → BACKGROUND EXECUTION
reach-preview
sizing how far an agent has reached or could reach, shown to the reviewer before or during the run → REACH PREVIEW
recursive self-improvement
a process where an AI model is used to build its own successor, potentially speeding up beyond human oversight → EMERGENCY STOP
register
the tone and level of detail an AI uses in a reply → CONFIDENCE SIGNALING
repositories
stored collections of code and its history, where developers track every change over time → AGENT IDENTITY
residual human states
the cognitive, emotional, and behavioral traces that stay with you after you finish working with an AI system
responsible-disclosure chain
the agreed process where a researcher tells a software maintainer about a bug privately before going public
reuse surface
a screen where a worker can take a past run and turn it into a starting point for the next task → DELEGATION
review surface
the document or view a reviewer actually sees when judging what an agent did → REVIEW
revision history
the record a wiki keeps of every edit, who made it, and when → OBSERVABILITY
revocation path
a clear way for a user to remove the AI's access at any time → PERMISSIONS
reward-hacking
agents finding shortcuts to score well on a goal without doing what the goal intended → OFF-BRIEF ALERT
routing servers
servers that pass requests from an AI agent out to websites and services → APPROVAL
rubric
a list of rules used to score an answer, with a pass or fail for each rule → EVAL SET AUTHORING
runtime
the moment the agent is running and taking actions, as opposed to the setup stage → APPROVAL
Rust
Rust, a programming language valued for speed and memory safety

S

sanctioned scope
the tasks and actions a human has approved an agent to carry out → PERMISSIONS
sandbox
a contained environment where the agent is meant to operate, separate from outside systems → PERMISSIONS
sandboxed capability set
a contained collection of tools the agent can use, kept separate from the rest of your computer → PERMISSIONS
sandboxing
running the agent in a contained environment cut off from the rest of your system → PERMISSIONS
scaffolding
the supporting code and tools that let a reviewer inspect and test what an agent did
schema
a formal description of what inputs a tool expects and what it returns → MEMORY
scoped permissions
limits placed on exactly which actions and data an AI is allowed to touch → PERMISSIONS
scorer
a function that grades AI outputs against a standard you define → EVAL SET AUTHORING
screen reader
software that reads the screen aloud for blind users
SDK
software development kit, a set of pre-built code that helps engineers build on a platform faster
semantic collapse
the slow shrinking of an agent's output variety until everything it produces feels the same → CONTEXT
SEP-1036
a numbered proposal that adds a new rule to the MCP standard
SEP-2322
a numbered draft proposal from the MCP Transports Working Group
shared artifact
a file or workspace that multiple agents can all access, read, and write to → AGENT-TO-AGENT
shared state
information that more than one agent can see and change at the same time → OBSERVABILITY
shared work state
a live record of work in progress that both a human and an AI agent can read and update together → CONTEXT
skill registries
stored lists of tools and actions the AI agent is allowed to call → PERMISSIONS
skill registry
the list of registered AI tools your agents can call, stored so the agent knows what actions are available → MEMORY
skill rot
when a registered tool drifts out of sync with the outside services it connects to, so it silently misbehaves → OBSERVABILITY
SPEAR
the five-principle framework the paper proposes for keeping humans in charge of AI agents
stacked defenses
multiple security measures layered on top of each other so that one failure does not let an attack through → PERMISSIONS
standards body
an independent group that sets shared rules an industry agrees to follow → OBSERVABILITY
stateful authorization
a permission model that tracks what the agent has already done, not just what it is allowed to do → PERMISSIONS
stopping criteria
the conditions that tell an agent when to stop acting and hand the work back → APPROVAL
structured data
information stored in a fixed, predictable format that software can read and display → GENERATIVE UI
sub-agent
a helper AI that a main AI spins up to handle a piece of a larger task → AGENT-TO-AGENT
swarm
a group of many AI helpers running at the same time → AGENT-TO-AGENT
system text
a setup instruction the platform writes before any conversation starts, which shapes how the AI behaves → CONTEXT

T

terminal
the text-only command window on your computer
test harness
the set of automated checks that run against an agent's output to confirm it is correct → EVAL RUN SETUP
test suite
the set of checks that confirm an AI coding agent's output is correct → EVAL RUN SETUP
test-retest agreement
how often a judge gives the same score when shown the same case twice → CASE DRILLDOWN
thicker interfaces
a design approach where the screen between you and an agent shows assumptions, surfaces disagreements, and holds a recor → PLANNING
token
the unit an AI reads and processes; providers charge by the number of tokens used → BUDGET SCOPING
token system
a set of named design values, such as colours and font sizes, that tools use to stay consistent → CONTEXT
tool enumeration
registering a fixed, named list of actions the AI is allowed to take → PERMISSIONS
tool registry
a server-side list of tools the agent is allowed to use, along with their descriptions → PERMISSIONS
TOOLS.md
a file that lists which tools and permissions the agent is allowed to use → PERMISSIONS
traces
timestamped logs of every action an AI agent took, so a human can review what happened → AGENT IDENTITY
training data
the existing text and code an AI model learned from before it was released → AGENT IDENTITY
trajectories
patterns of change that build up in a person across repeated interactions with an AI
transport layer
the low-level plumbing that carries messages between an app and a server
trust calibration
how accurately you judge when to trust the AI and when to question it → CONFIDENCE SIGNALING

U

UI-level interventions
changes to what users see on screen to help them understand and control the assistant → MEMORY
undominated listing
a listing that beats every other option on the page across the attributes shown → DELEGATION
URL Mode Elicitation
a built-in signal that tells the app to send the user to a separate web page for a sensitive step → APPROVAL

V

version-controlled
saved in a system that tracks every change, so you can see who changed what and roll back if needed → CONTEXT
vibecoding
building software or agent workflows informally, by writing prompts rather than formal specifications
vision model
an AI model that reads images, here used to look at the iPhone screen and decide what to tap
vision-language model
an AI model that reads both images and text

W

weighted rollout
a way of sending a share of real traffic to a new version while the old version still handles the rest → PRODUCTION DRIFT
whistleblower pattern
a design approach that lets an agent alert a human when it detects out-of-bounds behavior
workflow templates
saved step-by-step plans that tell an agent how to carry out a task → DELEGATION
working memory
the facts the assistant can currently read and act on in an ongoing conversation → MEMORY

X

x402 payment protocol
x402, a standard that lets a website charge an AI agent a small fee before serving content → BUDGET SCOPING