← Field Notes
NOV 4 · Paper · via arXiv Agent-to-AgentObservabilityReview

AI hiring screeners favor some AI writing models over others

One AI wrote the application. Another AI scored it. The reviewer saw only the number. A reviewer cannot question a score when nothing shows them that the scoring tool preferred one writing tool over another.

Machine summary of the source

A new study finds that AI screeners used in hiring show a consistent bias toward job applications written by certain AI models. The bias holds regardless of the actual quality of the candidate's experience or skills. That means two candidates with identical backgrounds can receive different scores based on which AI tool helped them write their cover letter.

The researchers tested multiple AI writing models and multiple AI screeners. Some model pairings produced scores far above the median. Others were penalized. The candidates had no way to know this was happening, and the screeners gave no signal that model identity was affecting their judgment.

For teams designing hiring tools, or any workflow where one AI produces content that another AI judges, this exposes a blind spot. The human reviewing the final score sees a number, not the chain of decisions behind it. If the AI judge carries a hidden preference for certain AI writers, the human has no basis to question it. That is the design problem: a person cannot correct a bias they cannot see.

The summary above is generated; the note at the top is the editorial judgment. Primary source ↗